Note: This is test shard 3 of 3.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/2
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260620 18:31:28.735100 10143 test_util.cc:268] Using random seed: -1642375367
2026-06-20T18:31:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:31:28Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskkm04Ww/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskkm04Ww/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/security.properties
2026-06-20 18:31:36: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskkm04Ww/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskkm04Ww/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260620 18:31:45.207001 10143 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
/tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.254:43203
--webserver_interface=127.9.231.254
--webserver_port=0
--builtin_ntp_servers=127.9.231.212:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260620 18:31:45.625133 10385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:31:45.625555 10385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:31:45.625703 10385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:31:45.634276 10385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:31:45.634438 10385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:31:45.634490 10385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:31:45.634618 10385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:31:45.645663 10385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.231.212:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.254:43203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.231.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 17:42:20 UTC on ccycloud.quasar-rymxqh.root.comops.site
build id 4457681
ASAN enabled
I20260620 18:31:45.647660 10385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:31:45.650238 10385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:31:45.666643 10391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:45.667083 10390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:45.666728 10393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:31:45.668089 10385 server_base.cc:1034] running on GCE node
I20260620 18:31:45.669550 10385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:31:45.672439 10385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:31:45.673858 10385 hybrid_clock.cc:648] HybridClock initialized: now 1781980305673717 us; error 116 us; skew 500 ppm
I20260620 18:31:45.674636 10385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 18:31:45.682536 10385 webserver.cc:460] Webserver started at http://127.9.231.254:45071/ using document root <none> and password file <none>
I20260620 18:31:45.683665 10385 fs_manager.cc:359] Metadata directory not provided
I20260620 18:31:45.683825 10385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:31:45.684226 10385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:31:45.690100 10385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data/instance:
uuid: "1cd216a992124f0fac2fb65754e6f7df"
format_stamp: "Formatted at 2026-06-20 18:31:45 on dist-test-slave-dv4w"
I20260620 18:31:45.691357 10385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal/instance:
uuid: "1cd216a992124f0fac2fb65754e6f7df"
format_stamp: "Formatted at 2026-06-20 18:31:45 on dist-test-slave-dv4w"
I20260620 18:31:45.700305 10385 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.001s
I20260620 18:31:45.705860 10399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:31:45.708109 10385 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.001s
I20260620 18:31:45.708490 10385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data,/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
uuid: "1cd216a992124f0fac2fb65754e6f7df"
format_stamp: "Formatted at 2026-06-20 18:31:45 on dist-test-slave-dv4w"
I20260620 18:31:45.708847 10385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:31:45.729878 10385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:31:45.731087 10385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:31:45.731501 10385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:31:45.758086 10385 rpc_server.cc:307] RPC server started. Bound to: 127.9.231.254:43203
I20260620 18:31:45.758107 10450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.231.254:43203 every 8 connection(s)
I20260620 18:31:45.760082 10385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data/info.pb
I20260620 18:31:45.763952 10143 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu as pid 10385
I20260620 18:31:45.764389 10143 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal/instance
I20260620 18:31:45.766258 10451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:45.768692 10143 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
/tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.253:44633
--webserver_interface=127.9.231.253
--webserver_port=0
--builtin_ntp_servers=127.9.231.212:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260620 18:31:45.777607 10451 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:31:45.811249 10451 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } has no permanent_uuid. Determining permanent_uuid...
W20260620 18:31:45.812870 10402 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:31:45.815351 10451 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:31:45.864779 10451 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } attempt: 1
W20260620 18:31:45.867693 10451 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:31:45.936115 10451 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } attempt: 2
W20260620 18:31:45.938802 10451 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:31:46.030251 10451 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } attempt: 3
W20260620 18:31:46.033053 10451 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:31:46.176455 10451 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } attempt: 4
W20260620 18:31:46.178682 10451 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:31:46.192158 10452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:31:46.192654 10452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:31:46.192817 10452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:31:46.201803 10452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:31:46.201969 10452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:31:46.202016 10452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:31:46.202070 10452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:31:46.212781 10452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.231.212:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.253:44633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.231.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 17:42:20 UTC on ccycloud.quasar-rymxqh.root.comops.site
build id 4457681
ASAN enabled
I20260620 18:31:46.214696 10452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:31:46.217226 10452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:31:46.231933 10460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:46.231843 10461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:46.231848 10463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:31:46.233520 10452 server_base.cc:1034] running on GCE node
I20260620 18:31:46.234623 10452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:31:46.236766 10452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:31:46.238075 10452 hybrid_clock.cc:648] HybridClock initialized: now 1781980306237989 us; error 51 us; skew 500 ppm
I20260620 18:31:46.238600 10452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 18:31:46.241694 10452 webserver.cc:460] Webserver started at http://127.9.231.253:37199/ using document root <none> and password file <none>
I20260620 18:31:46.242599 10452 fs_manager.cc:359] Metadata directory not provided
I20260620 18:31:46.242718 10452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:31:46.243070 10452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:31:46.246485 10452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data/instance:
uuid: "650e0197335644718cd91d5018399ee2"
format_stamp: "Formatted at 2026-06-20 18:31:46 on dist-test-slave-dv4w"
I20260620 18:31:46.247427 10452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal/instance:
uuid: "650e0197335644718cd91d5018399ee2"
format_stamp: "Formatted at 2026-06-20 18:31:46 on dist-test-slave-dv4w"
I20260620 18:31:46.254102 10452 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260620 18:31:46.259069 10469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:31:46.261402 10452 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260620 18:31:46.261698 10452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data,/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
uuid: "650e0197335644718cd91d5018399ee2"
format_stamp: "Formatted at 2026-06-20 18:31:46 on dist-test-slave-dv4w"
I20260620 18:31:46.261981 10452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:31:46.290616 10452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:31:46.291983 10452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:31:46.292456 10452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:31:46.320190 10452 rpc_server.cc:307] RPC server started. Bound to: 127.9.231.253:44633
I20260620 18:31:46.320197 10520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.231.253:44633 every 8 connection(s)
I20260620 18:31:46.322314 10452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data/info.pb
I20260620 18:31:46.327041 10521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:46.328195 10143 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu as pid 10452
I20260620 18:31:46.328400 10143 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal/instance
I20260620 18:31:46.329928 10143 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
/tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.252:34267
--webserver_interface=127.9.231.252
--webserver_port=0
--builtin_ntp_servers=127.9.231.212:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260620 18:31:46.336711 10521 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:31:46.354583 10521 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:31:46.365947 10521 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } has no permanent_uuid. Determining permanent_uuid...
W20260620 18:31:46.367959 10472 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:31:46.370098 10521 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.231.252:34267: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:31:46.419574 10521 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } attempt: 1
W20260620 18:31:46.422082 10521 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.231.252:34267: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:31:46.478144 10451 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } attempt: 5
I20260620 18:31:46.487684 10451 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } has no permanent_uuid. Determining permanent_uuid...
W20260620 18:31:46.489676 10451 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.231.252:34267: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:31:46.490381 10521 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } attempt: 2
W20260620 18:31:46.492424 10521 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.231.252:34267: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:31:46.541090 10451 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } attempt: 1
W20260620 18:31:46.544245 10451 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.231.252:34267: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:31:46.583784 10521 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } attempt: 3
W20260620 18:31:46.586330 10521 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.231.252:34267: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:31:46.612720 10451 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } attempt: 2
W20260620 18:31:46.615875 10451 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.231.252:34267: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:31:46.722184 10451 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } attempt: 3
W20260620 18:31:46.724795 10451 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.231.252:34267: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:31:46.729707 10521 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } attempt: 4
W20260620 18:31:46.732200 10521 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.231.252:34267: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:31:46.737795 10522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:31:46.738209 10522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:31:46.738305 10522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:31:46.747848 10522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:31:46.747989 10522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:31:46.748035 10522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:31:46.748143 10522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:31:46.760210 10522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.231.212:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.252:34267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.231.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 17:42:20 UTC on ccycloud.quasar-rymxqh.root.comops.site
build id 4457681
ASAN enabled
I20260620 18:31:46.762151 10522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:31:46.764621 10522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:31:46.780597 10532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:46.780807 10531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:46.780751 10534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:31:46.782385 10522 server_base.cc:1034] running on GCE node
I20260620 18:31:46.783656 10522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:31:46.785801 10522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:31:46.787150 10522 hybrid_clock.cc:648] HybridClock initialized: now 1781980306787041 us; error 83 us; skew 500 ppm
I20260620 18:31:46.787617 10522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 18:31:46.790922 10522 webserver.cc:460] Webserver started at http://127.9.231.252:42823/ using document root <none> and password file <none>
I20260620 18:31:46.791862 10522 fs_manager.cc:359] Metadata directory not provided
I20260620 18:31:46.791986 10522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:31:46.792436 10522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:31:46.796131 10522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data/instance:
uuid: "4587918520d849eaad95e35037e36608"
format_stamp: "Formatted at 2026-06-20 18:31:46 on dist-test-slave-dv4w"
I20260620 18:31:46.797152 10522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/wal/instance:
uuid: "4587918520d849eaad95e35037e36608"
format_stamp: "Formatted at 2026-06-20 18:31:46 on dist-test-slave-dv4w"
I20260620 18:31:46.804127 10522 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260620 18:31:46.809123 10540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:31:46.811084 10522 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260620 18:31:46.811364 10522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data,/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/wal
uuid: "4587918520d849eaad95e35037e36608"
format_stamp: "Formatted at 2026-06-20 18:31:46 on dist-test-slave-dv4w"
I20260620 18:31:46.811599 10522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:31:46.847151 10522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:31:46.848429 10522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:31:46.848769 10522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:31:46.874920 10591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.231.252:34267 every 8 connection(s)
I20260620 18:31:46.874933 10522 rpc_server.cc:307] RPC server started. Bound to: 127.9.231.252:34267
I20260620 18:31:46.877131 10522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data/info.pb
I20260620 18:31:46.883005 10592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:46.885401 10143 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu as pid 10522
I20260620 18:31:46.885591 10143 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/wal/instance
I20260620 18:31:46.892760 10592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:31:46.902096 10451 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } attempt: 4
I20260620 18:31:46.929226 10592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:31:46.937389 10451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Bootstrap starting.
I20260620 18:31:46.940830 10592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:31:46.941941 10451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:46.943410 10451 log.cc:824] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Log is configured to *not* fsync() on all Append() calls
I20260620 18:31:46.949455 10451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: No bootstrap required, opened a new log
I20260620 18:31:46.955742 10592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Bootstrap starting.
I20260620 18:31:46.958705 10451 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:46.959162 10451 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:46.959297 10451 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cd216a992124f0fac2fb65754e6f7df, State: Initialized, Role: FOLLOWER
I20260620 18:31:46.960693 10451 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:46.961488 10592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:46.962237 10600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:46.962811 10600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:46.963483 10592 log.cc:824] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Log is configured to *not* fsync() on all Append() calls
I20260620 18:31:46.963634 10451 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:31:46.969229 10592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: No bootstrap required, opened a new log
I20260620 18:31:46.981691 10592 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:46.982390 10592 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:46.982584 10592 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4587918520d849eaad95e35037e36608, State: Initialized, Role: FOLLOWER
I20260620 18:31:46.983664 10592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
W20260620 18:31:46.985798 10405 tablet.cc:2369] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 18:31:46.993358 10615 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:31:46.993628 10615 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:31:46.995344 10617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:46.996090 10617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:46.997376 10592 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:31:46.983858 10385 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: OK, instance_id { permanent_uuid: "650e0197335644718cd91d5018399ee2" instance_seqno: 1781980306313038 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 18:31:47.000802 10385 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:31:47.031514 10521 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } attempt: 5
I20260620 18:31:47.049942 10521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Bootstrap starting.
I20260620 18:31:47.054983 10521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:47.059783 10617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:47.060194 10617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:47.062139 10617 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
I20260620 18:31:47.062250 10521 log.cc:824] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Log is configured to *not* fsync() on all Append() calls
I20260620 18:31:47.064035 10426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cd216a992124f0fac2fb65754e6f7df" is_pre_election: true
I20260620 18:31:47.065111 10426 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4587918520d849eaad95e35037e36608 in term 0.
I20260620 18:31:47.066197 10542 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608; no voters:
I20260620 18:31:47.067528 10617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:31:47.067687 10617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:31:47.067785 10617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:47.072818 10617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:47.075157 10426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cd216a992124f0fac2fb65754e6f7df"
I20260620 18:31:47.075475 10426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:47.079910 10426 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4587918520d849eaad95e35037e36608 in term 1.
I20260620 18:31:47.085537 10542 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608; no voters:
I20260620 18:31:47.091853 10617 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 1 election: Requested vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
I20260620 18:31:47.092785 10617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:31:47.093525 10617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 1 LEADER]: Becoming Leader. State: Replica: 4587918520d849eaad95e35037e36608, State: Running, Role: LEADER
I20260620 18:31:47.094825 10617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
W20260620 18:31:47.101531 10546 tablet.cc:2369] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:31:47.115523 10635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4587918520d849eaad95e35037e36608. Latest consensus state: current_term: 1 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:47.116665 10635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: LEADER
I20260620 18:31:47.123610 10521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: No bootstrap required, opened a new log
I20260620 18:31:47.130272 10637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:31:47.154829 10637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:31:47.152385 10522 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: OK, instance_id { permanent_uuid: "650e0197335644718cd91d5018399ee2" instance_seqno: 1781980306313038 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 18:31:47.180750 10522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:31:47.207844 10426 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 1 FOLLOWER]: Refusing update from remote peer 4587918520d849eaad95e35037e36608: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:31:47.209218 10635 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:31:47.234094 10521 raft_consensus.cc:348] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:47.240469 10521 raft_consensus.cc:374] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:47.240813 10521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 650e0197335644718cd91d5018399ee2, State: Initialized, Role: FOLLOWER
I20260620 18:31:47.242038 10521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:47.248119 10600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4587918520d849eaad95e35037e36608. Latest consensus state: current_term: 1 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:47.248596 10600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:47.250488 10642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:47.252297 10639 mvcc.cc:205] Tried to move back new op lower bound from 7298991338305253376 to 7298991337920036864. Current Snapshot: MvccSnapshot[applied={T|T < 7298991338305253376}]
I20260620 18:31:47.257525 10617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:47.257923 10617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: LEADER
I20260620 18:31:47.263325 10642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:47.260843 10494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:47.268074 10494 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 1 FOLLOWER]: Refusing update from remote peer 4587918520d849eaad95e35037e36608: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:31:47.261067 10495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "650e0197335644718cd91d5018399ee2"
I20260620 18:31:47.268591 10637 catalog_manager.cc:1347] Generated new cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:31:47.268743 10637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:31:47.261425 10496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "650e0197335644718cd91d5018399ee2" is_pre_election: true
I20260620 18:31:47.273757 10496 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4587918520d849eaad95e35037e36608 in term 1.
I20260620 18:31:47.273950 10635 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [LEADER]: Connected to new peer: Peer: permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:31:47.274638 10600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:47.274922 10600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:47.259999 10521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: configured and running, proceeding with master startup.
W20260620 18:31:47.298874 10475 tablet.cc:2369] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:31:47.331532 10637 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:31:47.333729 10637 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:31:47.333581 10642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4587918520d849eaad95e35037e36608. Latest consensus state: current_term: 1 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:47.333990 10642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:47.334594 10649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:47.335039 10649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:47.336522 10617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:47.336833 10617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: LEADER
I20260620 18:31:47.359689 10637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Generated new TSK 0
I20260620 18:31:47.360987 10637 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:31:47.362481 10637 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260620 18:31:47.363996 10637 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260620 18:31:47.391328 10143 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
/tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.231.193:0
--local_ip_for_outbound_sockets=127.9.231.193
--webserver_interface=127.9.231.193
--webserver_port=0
--tserver_master_addrs=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--builtin_ntp_servers=127.9.231.212:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260620 18:31:47.396802 10664 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 18:31:47.860644 10669 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:31:47.860983 10669 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:47.863138 10670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:31:47.863529 10669 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
I20260620 18:31:47.863473 10670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
W20260620 18:31:47.863795 10665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:31:47.864328 10665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:31:47.864530 10665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260620 18:31:47.864639 10494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1cd216a992124f0fac2fb65754e6f7df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "650e0197335644718cd91d5018399ee2" is_pre_election: true
I20260620 18:31:47.864472 10567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1cd216a992124f0fac2fb65754e6f7df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4587918520d849eaad95e35037e36608" is_pre_election: true
I20260620 18:31:47.864954 10494 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cd216a992124f0fac2fb65754e6f7df in term 1.
I20260620 18:31:47.866537 10670 leader_election.cc:288] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
I20260620 18:31:47.866472 10402 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2; no voters: 4587918520d849eaad95e35037e36608
I20260620 18:31:47.866537 10425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "650e0197335644718cd91d5018399ee2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1cd216a992124f0fac2fb65754e6f7df" is_pre_election: true
I20260620 18:31:47.866878 10425 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 650e0197335644718cd91d5018399ee2 in term 1.
I20260620 18:31:47.867559 10471 leader_election.cc:302] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2; no voters:
I20260620 18:31:47.867944 10669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 18:31:47.868165 10669 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:31:47.868408 10669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 1 FOLLOWER]: Advancing to term 2
I20260620 18:31:47.868340 10567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "650e0197335644718cd91d5018399ee2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4587918520d849eaad95e35037e36608" is_pre_election: true
I20260620 18:31:47.869772 10670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 18:31:47.870005 10670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:31:47.870214 10670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 1 FOLLOWER]: Advancing to term 2
I20260620 18:31:47.874953 10670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:47.876837 10425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "650e0197335644718cd91d5018399ee2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1cd216a992124f0fac2fb65754e6f7df"
I20260620 18:31:47.878041 10567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "650e0197335644718cd91d5018399ee2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4587918520d849eaad95e35037e36608"
I20260620 18:31:47.879138 10670 leader_election.cc:288] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 2 election: Requested vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:31:47.879642 10665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:31:47.879974 10665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.231.193
I20260620 18:31:47.887594 10669 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:47.888475 10425 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 650e0197335644718cd91d5018399ee2 in current term 2: Already voted for candidate 1cd216a992124f0fac2fb65754e6f7df in this term.
I20260620 18:31:47.889823 10471 leader_election.cc:302] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 650e0197335644718cd91d5018399ee2; no voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608
I20260620 18:31:47.890208 10670 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260620 18:31:47.891016 10669 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 2 election: Requested vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
I20260620 18:31:47.891842 10494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1cd216a992124f0fac2fb65754e6f7df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "650e0197335644718cd91d5018399ee2"
I20260620 18:31:47.891989 10567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1cd216a992124f0fac2fb65754e6f7df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4587918520d849eaad95e35037e36608"
I20260620 18:31:47.892239 10494 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1cd216a992124f0fac2fb65754e6f7df in current term 2: Already voted for candidate 650e0197335644718cd91d5018399ee2 in this term.
I20260620 18:31:47.892735 10425 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 2 FOLLOWER]: Rejecting Update request from peer 4587918520d849eaad95e35037e36608 for earlier term 1. Current term is 2. Ops: []
I20260620 18:31:47.892900 10402 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:31:47.893453 10669 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260620 18:31:47.893456 10671 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260620 18:31:47.894512 10672 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 1 LEADER]: Stepping down as leader of term 1
I20260620 18:31:47.894663 10672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4587918520d849eaad95e35037e36608, State: Running, Role: LEADER
I20260620 18:31:47.895166 10672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:47.895903 10672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 1 FOLLOWER]: Advancing to term 2
I20260620 18:31:47.898046 10665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.231.212:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.231.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.231.193
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.231.193
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 17:42:20 UTC on ccycloud.quasar-rymxqh.root.comops.site
build id 4457681
ASAN enabled
I20260620 18:31:47.900427 10665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:31:47.902822 10665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:31:47.921094 10674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:47.921233 10675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:47.921180 10677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:31:47.923200 10665 server_base.cc:1034] running on GCE node
I20260620 18:31:47.924203 10665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:31:47.925733 10665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:31:47.927030 10665 hybrid_clock.cc:648] HybridClock initialized: now 1781980307926946 us; error 46 us; skew 500 ppm
I20260620 18:31:47.927440 10665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 18:31:47.936340 10665 webserver.cc:460] Webserver started at http://127.9.231.193:37491/ using document root <none> and password file <none>
I20260620 18:31:47.937507 10665 fs_manager.cc:359] Metadata directory not provided
I20260620 18:31:47.937674 10665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:31:47.938263 10665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:31:47.943259 10665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/data/instance:
uuid: "7696506dcc9746d0aea9b1129a3e9433"
format_stamp: "Formatted at 2026-06-20 18:31:47 on dist-test-slave-dv4w"
I20260620 18:31:47.944644 10665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal/instance:
uuid: "7696506dcc9746d0aea9b1129a3e9433"
format_stamp: "Formatted at 2026-06-20 18:31:47 on dist-test-slave-dv4w"
I20260620 18:31:47.953336 10665 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260620 18:31:47.959137 10683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:31:47.962031 10665 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260620 18:31:47.962303 10665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal
uuid: "7696506dcc9746d0aea9b1129a3e9433"
format_stamp: "Formatted at 2026-06-20 18:31:47 on dist-test-slave-dv4w"
I20260620 18:31:47.962586 10665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:31:47.984019 10665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:31:47.985061 10665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:31:47.985430 10665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:31:47.986637 10665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:31:47.988497 10665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:31:47.988615 10665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:31:47.988747 10665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:31:47.988838 10665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:31:47.997236 10615 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:31:47.997535 10615 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: loading cluster ID for follower catalog manager: success
I20260620 18:31:48.003278 10615 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: acquiring CA information for follower catalog manager: success
I20260620 18:31:48.007596 10615 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:31:48.035066 10665 rpc_server.cc:307] RPC server started. Bound to: 127.9.231.193:41961
I20260620 18:31:48.035358 10795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.231.193:41961 every 8 connection(s)
I20260620 18:31:48.036665 10665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/data/info.pb
I20260620 18:31:48.044148 10143 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu as pid 10665
I20260620 18:31:48.044467 10143 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal/instance
I20260620 18:31:48.081818 10797 heartbeater.cc:347] Connected to a master server at 127.9.231.254:43203
I20260620 18:31:48.082273 10797 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:48.083582 10797 heartbeater.cc:511] Master 127.9.231.254:43203 requested a full tablet report, sending...
I20260620 18:31:48.084789 10796 heartbeater.cc:347] Connected to a master server at 127.9.231.252:34267
I20260620 18:31:48.084996 10796 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:48.085161 10798 heartbeater.cc:347] Connected to a master server at 127.9.231.253:44633
I20260620 18:31:48.085407 10798 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:48.085565 10796 heartbeater.cc:511] Master 127.9.231.252:34267 requested a full tablet report, sending...
I20260620 18:31:48.086109 10798 heartbeater.cc:511] Master 127.9.231.253:44633 requested a full tablet report, sending...
I20260620 18:31:48.087000 10416 ts_manager.cc:194] Registered new tserver with Master: 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961)
I20260620 18:31:48.087704 10556 ts_manager.cc:194] Registered new tserver with Master: 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961)
I20260620 18:31:48.090236 10486 ts_manager.cc:194] Registered new tserver with Master: 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961)
I20260620 18:31:48.153151 10634 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:31:48.403075 10664 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:31:48.403237 10664 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: loading cluster ID for follower catalog manager: success
I20260620 18:31:48.406747 10664 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: acquiring CA information for follower catalog manager: success
I20260620 18:31:48.409358 10664 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:31:48.429194 10807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:48.429463 10807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:48.430421 10807 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
I20260620 18:31:48.431264 10425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1cd216a992124f0fac2fb65754e6f7df" is_pre_election: true
I20260620 18:31:48.431388 10494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "650e0197335644718cd91d5018399ee2" is_pre_election: true
I20260620 18:31:48.431627 10425 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4587918520d849eaad95e35037e36608 in term 2.
I20260620 18:31:48.431763 10494 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4587918520d849eaad95e35037e36608 in term 2.
I20260620 18:31:48.432307 10542 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608; no voters:
I20260620 18:31:48.432718 10807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260620 18:31:48.432875 10807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:31:48.433027 10807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 2 FOLLOWER]: Advancing to term 3
I20260620 18:31:48.435484 10807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:48.436347 10807 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 3 election: Requested vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
I20260620 18:31:48.437006 10425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1cd216a992124f0fac2fb65754e6f7df"
I20260620 18:31:48.437148 10494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "650e0197335644718cd91d5018399ee2"
I20260620 18:31:48.437305 10425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 2 FOLLOWER]: Advancing to term 3
I20260620 18:31:48.437451 10494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 2 FOLLOWER]: Advancing to term 3
I20260620 18:31:48.440779 10425 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4587918520d849eaad95e35037e36608 in term 3.
I20260620 18:31:48.440985 10494 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4587918520d849eaad95e35037e36608 in term 3.
I20260620 18:31:48.441413 10542 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608; no voters:
I20260620 18:31:48.442135 10807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 3 FOLLOWER]: Leader election won for term 3
I20260620 18:31:48.442996 10807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 3 LEADER]: Becoming Leader. State: Replica: 4587918520d849eaad95e35037e36608, State: Running, Role: LEADER
I20260620 18:31:48.443423 10807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:48.445225 10808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4587918520d849eaad95e35037e36608. Latest consensus state: current_term: 3 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:48.445663 10808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: LEADER
I20260620 18:31:48.446627 10810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:31:48.449782 10810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:31:48.451084 10810 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:31:48.451193 10810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:31:48.453006 10810 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:31:48.454092 10810 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Loaded TSK: 0
I20260620 18:31:48.455044 10810 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:31:48.456035 10810 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260620 18:31:48.457041 10810 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260620 18:31:48.473265 10143 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
/tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.231.194:0
--local_ip_for_outbound_sockets=127.9.231.194
--webserver_interface=127.9.231.194
--webserver_port=0
--tserver_master_addrs=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--builtin_ntp_servers=127.9.231.212:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260620 18:31:48.592692 10556 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.231.193:40039
W20260620 18:31:48.905583 10811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:31:48.906051 10811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:31:48.906240 10811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:31:48.915139 10811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:31:48.915453 10811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.231.194
I20260620 18:31:48.926060 10811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.231.212:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.231.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.231.194
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.231.194
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 17:42:20 UTC on ccycloud.quasar-rymxqh.root.comops.site
build id 4457681
ASAN enabled
I20260620 18:31:48.927919 10811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:31:48.930166 10811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:31:48.947108 10816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:48.948069 10819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:48.948083 10817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:31:48.949010 10811 server_base.cc:1034] running on GCE node
I20260620 18:31:48.949716 10811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:31:48.950882 10811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:31:48.952446 10811 hybrid_clock.cc:648] HybridClock initialized: now 1781980308952195 us; error 80 us; skew 500 ppm
I20260620 18:31:48.952940 10811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 18:31:48.956048 10811 webserver.cc:460] Webserver started at http://127.9.231.194:40969/ using document root <none> and password file <none>
I20260620 18:31:48.956943 10811 fs_manager.cc:359] Metadata directory not provided
I20260620 18:31:48.957059 10811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:31:48.957403 10811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:31:48.960465 10811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/data/instance:
uuid: "a0ae894b0758446aa6219aaa5267d1b4"
format_stamp: "Formatted at 2026-06-20 18:31:48 on dist-test-slave-dv4w"
I20260620 18:31:48.961265 10811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/wal/instance:
uuid: "a0ae894b0758446aa6219aaa5267d1b4"
format_stamp: "Formatted at 2026-06-20 18:31:48 on dist-test-slave-dv4w"
I20260620 18:31:48.966850 10825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:48.967015 10825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:48.967942 10825 leader_election.cc:288] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
I20260620 18:31:48.968945 10811 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260620 18:31:48.968935 10425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "650e0197335644718cd91d5018399ee2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1cd216a992124f0fac2fb65754e6f7df" is_pre_election: true
I20260620 18:31:48.969029 10567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "650e0197335644718cd91d5018399ee2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4587918520d849eaad95e35037e36608" is_pre_election: true
I20260620 18:31:48.969293 10425 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 650e0197335644718cd91d5018399ee2 in term 3.
I20260620 18:31:48.969878 10472 leader_election.cc:302] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2; no voters: 4587918520d849eaad95e35037e36608
I20260620 18:31:48.970258 10825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260620 18:31:48.970391 10825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:31:48.970532 10825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 3 FOLLOWER]: Advancing to term 4
I20260620 18:31:48.973088 10825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:48.974107 10825 leader_election.cc:288] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 4 election: Requested vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
I20260620 18:31:48.974884 10567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "650e0197335644718cd91d5018399ee2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4587918520d849eaad95e35037e36608"
I20260620 18:31:48.974853 10425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "650e0197335644718cd91d5018399ee2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1cd216a992124f0fac2fb65754e6f7df"
I20260620 18:31:48.975253 10425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 3 FOLLOWER]: Advancing to term 4
I20260620 18:31:48.975279 10826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:31:48.977747 10811 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260620 18:31:48.977985 10811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/wal
uuid: "a0ae894b0758446aa6219aaa5267d1b4"
format_stamp: "Formatted at 2026-06-20 18:31:48 on dist-test-slave-dv4w"
I20260620 18:31:48.978178 10425 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 650e0197335644718cd91d5018399ee2 in term 4.
I20260620 18:31:48.978271 10811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:31:48.979414 10471 leader_election.cc:302] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2; no voters: 4587918520d849eaad95e35037e36608
I20260620 18:31:48.979889 10825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 4 FOLLOWER]: Leader election won for term 4
I20260620 18:31:48.980585 10825 raft_consensus.cc:686] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 4 LEADER]: Becoming Leader. State: Replica: 650e0197335644718cd91d5018399ee2, State: Running, Role: LEADER
I20260620 18:31:48.981200 10825 consensus_queue.cc:237] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:48.983372 10827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 650e0197335644718cd91d5018399ee2. Latest consensus state: current_term: 4 leader_uuid: "650e0197335644718cd91d5018399ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:48.983973 10827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: LEADER
I20260620 18:31:48.984791 10829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:31:48.987655 10829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:31:48.988987 10829 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:31:48.989163 10829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:31:48.991299 10829 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:31:48.992543 10829 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Loaded TSK: 0
I20260620 18:31:48.993927 10829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:31:48.995190 10829 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260620 18:31:48.995229 10494 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 4 LEADER]: Rejecting Update request from peer 4587918520d849eaad95e35037e36608 for earlier term 3. Current term is 4. Ops: []
I20260620 18:31:48.996393 10829 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260620 18:31:48.996223 10834 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260620 18:31:48.996862 10834 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [LEADER]: Peer 650e0197335644718cd91d5018399ee2 log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260620 18:31:48.997044 10835 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 3 LEADER]: Stepping down as leader of term 3
I20260620 18:31:48.997202 10835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4587918520d849eaad95e35037e36608, State: Running, Role: LEADER
I20260620 18:31:48.997562 10835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:48.997967 10835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 3 FOLLOWER]: Advancing to term 4
I20260620 18:31:49.009321 10811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:31:49.010296 10811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:31:49.010653 10811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:31:49.011916 10811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:31:49.013911 10811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:31:49.014025 10811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:31:49.014179 10811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:31:49.014264 10811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:31:49.056602 10811 rpc_server.cc:307] RPC server started. Bound to: 127.9.231.194:45409
I20260620 18:31:49.056635 10943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.231.194:45409 every 8 connection(s)
I20260620 18:31:49.058076 10811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/data/info.pb
I20260620 18:31:49.059182 10143 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu as pid 10811
I20260620 18:31:49.059422 10143 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/wal/instance
I20260620 18:31:49.070058 10143 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
/tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.231.195:0
--local_ip_for_outbound_sockets=127.9.231.195
--webserver_interface=127.9.231.195
--webserver_port=0
--tserver_master_addrs=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--builtin_ntp_servers=127.9.231.212:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260620 18:31:49.096778 10796 heartbeater.cc:503] Master 127.9.231.252:34267 was elected leader, sending a full tablet report...
I20260620 18:31:49.099292 10945 heartbeater.cc:347] Connected to a master server at 127.9.231.254:43203
I20260620 18:31:49.099396 10944 heartbeater.cc:347] Connected to a master server at 127.9.231.252:34267
I20260620 18:31:49.100093 10945 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:49.100229 10944 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:49.101752 10945 heartbeater.cc:511] Master 127.9.231.254:43203 requested a full tablet report, sending...
I20260620 18:31:49.101876 10944 heartbeater.cc:511] Master 127.9.231.252:34267 requested a full tablet report, sending...
I20260620 18:31:49.103204 10946 heartbeater.cc:347] Connected to a master server at 127.9.231.253:44633
I20260620 18:31:49.103364 10946 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:49.103960 10946 heartbeater.cc:511] Master 127.9.231.253:44633 requested a full tablet report, sending...
I20260620 18:31:49.105275 10557 ts_manager.cc:194] Registered new tserver with Master: a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:31:49.105254 10416 ts_manager.cc:194] Registered new tserver with Master: a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:31:49.105473 10486 ts_manager.cc:194] Registered new tserver with Master: a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:31:49.107142 10486 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.231.194:33925
I20260620 18:31:49.365902 10425 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 4 FOLLOWER]: Refusing update from remote peer 650e0197335644718cd91d5018399ee2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260620 18:31:49.366870 10827 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260620 18:31:49.369880 10955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 650e0197335644718cd91d5018399ee2. Latest consensus state: current_term: 4 leader_uuid: "650e0197335644718cd91d5018399ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:49.370277 10955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:49.373960 10825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "650e0197335644718cd91d5018399ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:49.374401 10825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: LEADER
I20260620 18:31:49.374431 10955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "650e0197335644718cd91d5018399ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:49.374739 10955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:49.375402 10567 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 4 FOLLOWER]: Refusing update from remote peer 650e0197335644718cd91d5018399ee2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260620 18:31:49.375546 10567 pending_rounds.cc:76] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Aborting all ops after (but not including) 4
I20260620 18:31:49.375643 10567 pending_rounds.cc:97] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260620 18:31:49.375766 10567 raft_consensus.cc:2866] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260620 18:31:49.377148 10827 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260620 18:31:49.390079 10835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 650e0197335644718cd91d5018399ee2. Latest consensus state: current_term: 4 leader_uuid: "650e0197335644718cd91d5018399ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:49.390542 10835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:49.390722 10834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "650e0197335644718cd91d5018399ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:49.391083 10834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 18:31:49.526139 10954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:31:49.526608 10954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:31:49.526799 10954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:31:49.534976 10954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:31:49.535287 10954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.231.195
I20260620 18:31:49.545850 10954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.231.212:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.231.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.231.195
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.231.195
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 17:42:20 UTC on ccycloud.quasar-rymxqh.root.comops.site
build id 4457681
ASAN enabled
I20260620 18:31:49.547706 10954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:31:49.550040 10954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:31:49.567744 10965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:49.567802 10968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:49.568338 10966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:31:49.569099 10954 server_base.cc:1034] running on GCE node
I20260620 18:31:49.569993 10954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:31:49.571290 10954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:31:49.572602 10954 hybrid_clock.cc:648] HybridClock initialized: now 1781980309572478 us; error 73 us; skew 500 ppm
I20260620 18:31:49.573063 10954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 18:31:49.576409 10954 webserver.cc:460] Webserver started at http://127.9.231.195:40597/ using document root <none> and password file <none>
I20260620 18:31:49.577354 10954 fs_manager.cc:359] Metadata directory not provided
I20260620 18:31:49.577538 10954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:31:49.577963 10954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:31:49.581825 10954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/data/instance:
uuid: "9a4ce671d3984e1a8592bc29d45612bc"
format_stamp: "Formatted at 2026-06-20 18:31:49 on dist-test-slave-dv4w"
I20260620 18:31:49.582978 10954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal/instance:
uuid: "9a4ce671d3984e1a8592bc29d45612bc"
format_stamp: "Formatted at 2026-06-20 18:31:49 on dist-test-slave-dv4w"
I20260620 18:31:49.591017 10954 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260620 18:31:49.596722 10974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:31:49.597133 10798 heartbeater.cc:503] Master 127.9.231.253:44633 was elected leader, sending a full tablet report...
I20260620 18:31:49.599697 10954 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260620 18:31:49.599987 10954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal
uuid: "9a4ce671d3984e1a8592bc29d45612bc"
format_stamp: "Formatted at 2026-06-20 18:31:49 on dist-test-slave-dv4w"
I20260620 18:31:49.600328 10954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:31:49.611234 10946 heartbeater.cc:503] Master 127.9.231.253:44633 was elected leader, sending a full tablet report...
I20260620 18:31:49.630761 10954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:31:49.631979 10954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:31:49.632453 10954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:31:49.633802 10954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:31:49.635901 10954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:31:49.635986 10954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:31:49.636132 10954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:31:49.636199 10954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:31:49.678970 10954 rpc_server.cc:307] RPC server started. Bound to: 127.9.231.195:42583
I20260620 18:31:49.679118 11086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.231.195:42583 every 8 connection(s)
I20260620 18:31:49.680703 10954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/data/info.pb
I20260620 18:31:49.683571 10143 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu as pid 10954
I20260620 18:31:49.683813 10143 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal/instance
I20260620 18:31:49.712621 11089 heartbeater.cc:347] Connected to a master server at 127.9.231.253:44633
I20260620 18:31:49.712657 11088 heartbeater.cc:347] Connected to a master server at 127.9.231.254:43203
I20260620 18:31:49.713147 11088 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:49.713153 11089 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:49.713531 11087 heartbeater.cc:347] Connected to a master server at 127.9.231.252:34267
I20260620 18:31:49.713665 11087 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:49.714717 11089 heartbeater.cc:511] Master 127.9.231.253:44633 requested a full tablet report, sending...
I20260620 18:31:49.716192 11087 heartbeater.cc:511] Master 127.9.231.252:34267 requested a full tablet report, sending...
I20260620 18:31:49.716372 11088 heartbeater.cc:511] Master 127.9.231.254:43203 requested a full tablet report, sending...
I20260620 18:31:49.717598 10485 ts_manager.cc:194] Registered new tserver with Master: 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:49.718005 10415 ts_manager.cc:194] Registered new tserver with Master: 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:49.719374 10485 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.231.195:39557
I20260620 18:31:49.720489 10557 ts_manager.cc:194] Registered new tserver with Master: 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:49.741307 10143 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:31:49.780925 10482 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35900:
name: "default.Table_23d8d896ffe7406ca154449fd1a1eeab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:49.781138 10483 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35900:
name: "default.Table_08795a50aec84f85bc35ae8025674185"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:49.782435 10486 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35900:
name: "default.Table_9e9a3aec7a67407ba81fe1f6797b55f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:49.781056 10481 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35900:
name: "default.Table_f27c96a5d11944f6a75123e42b5b2ce5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:49.782722 10484 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35900:
name: "default.Table_be97363936284e2faa8400049925a98c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:49.782848 10480 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35900:
name: "default.Table_009e06deb1f84f74b4a32287a8c44b5a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:49.783834 10479 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35900:
name: "default.Table_269d7d1648864d4e8b29b944b7115e6a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:49.784700 10478 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35900:
name: "default.Table_aa7e29fa2b8e4974a8ece0e46aba222a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:49.785748 10477 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35900:
name: "default.Table_1ba08641833a48838fae05c1f72333e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:49.779639 10485 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35900:
name: "default.Table_88afa4cb37564293a220876026ee1931"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:31:49.812052 10481 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f27c96a5d11944f6a75123e42b5b2ce5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:31:49.814055 10143 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu pid 10452
W20260620 18:31:49.814261 10483 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_08795a50aec84f85bc35ae8025674185 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:31:49.815398 10516 generic_service.cc:196] Checking for leaks (request via RPC)
W20260620 18:31:49.816239 10479 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_269d7d1648864d4e8b29b944b7115e6a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260620 18:31:49.816568 10482 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_23d8d896ffe7406ca154449fd1a1eeab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260620 18:31:49.816847 10480 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_009e06deb1f84f74b4a32287a8c44b5a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260620 18:31:49.817410 10478 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_aa7e29fa2b8e4974a8ece0e46aba222a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260620 18:31:49.817965 10484 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_be97363936284e2faa8400049925a98c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260620 18:31:49.818239 10477 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1ba08641833a48838fae05c1f72333e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:31:49.879709 11137 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:31:49.880023 11137 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:49.880865 11137 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
I20260620 18:31:49.882158 10567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1cd216a992124f0fac2fb65754e6f7df" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4587918520d849eaad95e35037e36608" is_pre_election: true
I20260620 18:31:49.882460 10567 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cd216a992124f0fac2fb65754e6f7df in term 4.
I20260620 18:31:49.883210 10402 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608; no voters:
I20260620 18:31:49.883385 10834 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:31:49.883527 10834 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:49.883728 11137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260620 18:31:49.883881 11137 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 4 FOLLOWER]: Starting leader election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:31:49.883972 11137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 4 FOLLOWER]: Advancing to term 5
I20260620 18:31:49.884488 10834 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
I20260620 18:31:49.884984 10425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1cd216a992124f0fac2fb65754e6f7df" is_pre_election: true
I20260620 18:31:49.888049 11137 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:49.888520 10425 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4587918520d849eaad95e35037e36608 in current term 5: Already voted for candidate 1cd216a992124f0fac2fb65754e6f7df in this term.
I20260620 18:31:49.890573 10567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1cd216a992124f0fac2fb65754e6f7df" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4587918520d849eaad95e35037e36608"
I20260620 18:31:49.891502 10567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 4 FOLLOWER]: Advancing to term 5
I20260620 18:31:49.894891 10567 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cd216a992124f0fac2fb65754e6f7df in term 5.
I20260620 18:31:49.895584 10402 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608; no voters:
I20260620 18:31:49.901476 11137 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 5 election: Requested vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
I20260620 18:31:49.902103 11137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 5 FOLLOWER]: Leader election won for term 5
I20260620 18:31:49.902598 11137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 5 LEADER]: Becoming Leader. State: Replica: 1cd216a992124f0fac2fb65754e6f7df, State: Running, Role: LEADER
I20260620 18:31:49.903350 11137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:49.908586 11138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cd216a992124f0fac2fb65754e6f7df. Latest consensus state: current_term: 5 leader_uuid: "1cd216a992124f0fac2fb65754e6f7df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:49.909013 11138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: LEADER
I20260620 18:31:49.917012 11139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:31:49.919962 11139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:31:49.925758 11139 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:31:49.925979 11139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:31:49.927665 11139 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:31:49.928856 11139 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Loaded TSK: 0
I20260620 18:31:49.930157 11139 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:31:49.931200 11139 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260620 18:31:49.932379 11139 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
W20260620 18:31:50.015839 10485 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_88afa4cb37564293a220876026ee1931 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260620 18:31:50.016530 10486 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_9e9a3aec7a67407ba81fe1f6797b55f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:31:50.017207 10494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1cd216a992124f0fac2fb65754e6f7df" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "650e0197335644718cd91d5018399ee2" is_pre_election: true
I20260620 18:31:50.018849 10494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "650e0197335644718cd91d5018399ee2" is_pre_election: true
I20260620 18:31:50.019364 10827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "650e0197335644718cd91d5018399ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:50.019640 10827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: LEADER
I20260620 18:31:50.021775 10425 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 5 LEADER]: Rejecting Update request from peer 650e0197335644718cd91d5018399ee2 for earlier term 4. Current term is 5. Ops: []
I20260620 18:31:50.022698 10543 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4587918520d849eaad95e35037e36608; no voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2
I20260620 18:31:50.023378 10834 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260620 18:31:50.025434 10567 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 5 FOLLOWER]: Rejecting Update request from peer 650e0197335644718cd91d5018399ee2 for earlier term 4. Current term is 5. Ops: []
I20260620 18:31:50.026636 10825 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260620 18:31:50.028684 10143 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu with pid 10452
W20260620 18:31:50.058893 11099 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.231.253:44633)
I20260620 18:31:50.071295 10412 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36824:
name: "default.Table_9e9a3aec7a67407ba81fe1f6797b55f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:50.072472 10414 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36824:
name: "default.Table_269d7d1648864d4e8b29b944b7115e6a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:50.073052 10410 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36824:
name: "default.Table_be97363936284e2faa8400049925a98c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:50.073774 10413 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36824:
name: "default.Table_009e06deb1f84f74b4a32287a8c44b5a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:50.074159 10408 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36824:
name: "default.Table_23d8d896ffe7406ca154449fd1a1eeab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:50.075287 10407 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36824:
name: "default.Table_f27c96a5d11944f6a75123e42b5b2ce5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:50.081156 10411 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36824:
name: "default.Table_88afa4cb37564293a220876026ee1931"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:50.081871 10409 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36824:
name: "default.Table_08795a50aec84f85bc35ae8025674185"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:50.071511 10416 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36824:
name: "default.Table_aa7e29fa2b8e4974a8ece0e46aba222a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:50.085754 10415 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36824:
name: "default.Table_1ba08641833a48838fae05c1f72333e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:31:50.063230 10400 connection.cc:535] server connection from 127.0.0.1:36794 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:31:50.090085 11143 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:31:50.063863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.231.253:44633 (local address 127.0.0.1:35920)
0620 18:31:50.079218 (+ 15355us) negotiation.cc:107] Waiting for socket to connect
0620 18:31:50.079250 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 18:31:50.079531 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:31:50.088744 (+ 9213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":15157,"thread_start_us":14963,"threads_started":1}
W20260620 18:31:50.090809 11101 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:31:50.060580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.231.253:44633 (local address 127.0.0.1:35910)
0620 18:31:50.060984 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0620 18:31:50.061006 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 18:31:50.061194 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:31:50.088670 (+ 27476us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":122}
W20260620 18:31:50.090915 10402 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:31:50.091485 10412 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_9e9a3aec7a67407ba81fe1f6797b55f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260620 18:31:50.092770 10410 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_be97363936284e2faa8400049925a98c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260620 18:31:50.093336 10413 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_009e06deb1f84f74b4a32287a8c44b5a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260620 18:31:50.094113 10414 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_269d7d1648864d4e8b29b944b7115e6a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260620 18:31:50.092773 10408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_23d8d896ffe7406ca154449fd1a1eeab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260620 18:31:50.105688 10798 heartbeater.cc:644] Failed to heartbeat to 127.9.231.253:44633 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:31:50.117684 10946 heartbeater.cc:644] Failed to heartbeat to 127.9.231.253:44633 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:31:50.223325 11089 heartbeater.cc:503] Master 127.9.231.253:44633 was elected leader, sending a full tablet report...
W20260620 18:31:50.230391 11089 heartbeater.cc:644] Failed to heartbeat to 127.9.231.253:44633 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:31:50.326032 10567 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 5 FOLLOWER]: Refusing update from remote peer 1cd216a992124f0fac2fb65754e6f7df: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260620 18:31:50.326987 11138 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [LEADER]: Connected to new peer: Peer: permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260620 18:31:50.329125 10834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cd216a992124f0fac2fb65754e6f7df. Latest consensus state: current_term: 5 leader_uuid: "1cd216a992124f0fac2fb65754e6f7df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:50.329435 10834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:50.331542 11137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "1cd216a992124f0fac2fb65754e6f7df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:50.332027 11137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: LEADER
W20260620 18:31:50.333541 10402 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df -> Peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Couldn't send request to peer 650e0197335644718cd91d5018399ee2. Status: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 18:31:50.344198 10834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "1cd216a992124f0fac2fb65754e6f7df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:50.344532 10834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:50.453652 10414 catalog_manager.cc:2148] created HMS catalog entry for table [id=963ec1c8d90b48ac9b0124b31d53cfe3]
I20260620 18:31:50.483362 10414 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36824:
table { table_name: "default.Table_269d7d1648864d4e8b29b944b7115e6a" } new_table_name: "default.Table_45bc016e4a9b4288a42f33020e656b17" modify_external_catalogs: true
I20260620 18:31:50.521029 11022 tablet_service.cc:1467] Processing CreateTablet for tablet c865fa696725420ca2dec405247cf60c (DEFAULT_TABLE table=default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.523607 11022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c865fa696725420ca2dec405247cf60c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:50.533638 10879 tablet_service.cc:1467] Processing CreateTablet for tablet c865fa696725420ca2dec405247cf60c (DEFAULT_TABLE table=default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.535765 10879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c865fa696725420ca2dec405247cf60c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:50.537150 10413 catalog_manager.cc:2148] created HMS catalog entry for table [id=83dd7dd7bed24f758c98e9019c63ef7c]
I20260620 18:31:50.541929 10731 tablet_service.cc:1467] Processing CreateTablet for tablet c865fa696725420ca2dec405247cf60c (DEFAULT_TABLE table=default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.544333 10731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c865fa696725420ca2dec405247cf60c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:50.548475 10413 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36824:
table { table_name: "default.Table_009e06deb1f84f74b4a32287a8c44b5a" } new_table_name: "default.Table_7105b83c587d4cf0a6add4272b0defc6" modify_external_catalogs: true
I20260620 18:31:50.554451 10730 tablet_service.cc:1467] Processing CreateTablet for tablet 1d85fc3e284f48f080b205bfdcb55e6e (DEFAULT_TABLE table=default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.555068 10730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d85fc3e284f48f080b205bfdcb55e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:50.556643 10878 tablet_service.cc:1467] Processing CreateTablet for tablet 1d85fc3e284f48f080b205bfdcb55e6e (DEFAULT_TABLE table=default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.556619 11021 tablet_service.cc:1467] Processing CreateTablet for tablet 1d85fc3e284f48f080b205bfdcb55e6e (DEFAULT_TABLE table=default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.557196 10878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d85fc3e284f48f080b205bfdcb55e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:50.557271 11021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d85fc3e284f48f080b205bfdcb55e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:50.575157 11159 tablet_bootstrap.cc:492] T c865fa696725420ca2dec405247cf60c P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
I20260620 18:31:50.581393 11159 tablet_bootstrap.cc:654] T c865fa696725420ca2dec405247cf60c P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:50.582477 11157 tablet_bootstrap.cc:492] T c865fa696725420ca2dec405247cf60c P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
I20260620 18:31:50.583057 11159 log.cc:824] T c865fa696725420ca2dec405247cf60c P a0ae894b0758446aa6219aaa5267d1b4: Log is configured to *not* fsync() on all Append() calls
I20260620 18:31:50.585425 11158 tablet_bootstrap.cc:492] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:31:50.587320 11157 tablet_bootstrap.cc:654] T c865fa696725420ca2dec405247cf60c P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:50.588819 11159 tablet_bootstrap.cc:492] T c865fa696725420ca2dec405247cf60c P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:31:50.589170 11159 ts_tablet_manager.cc:1397] T c865fa696725420ca2dec405247cf60c P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260620 18:31:50.590224 11158 tablet_bootstrap.cc:654] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:50.590850 11157 log.cc:824] T c865fa696725420ca2dec405247cf60c P 9a4ce671d3984e1a8592bc29d45612bc: Log is configured to *not* fsync() on all Append() calls
I20260620 18:31:50.591672 11158 log.cc:824] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433: Log is configured to *not* fsync() on all Append() calls
I20260620 18:31:50.604625 11159 raft_consensus.cc:348] T c865fa696725420ca2dec405247cf60c P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:50.605480 11159 raft_consensus.cc:374] T c865fa696725420ca2dec405247cf60c P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.605599 11159 raft_consensus.cc:729] T c865fa696725420ca2dec405247cf60c P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:31:50.609884 11159 consensus_queue.cc:260] T c865fa696725420ca2dec405247cf60c P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:50.615330 11158 tablet_bootstrap.cc:492] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:31:50.615731 11158 ts_tablet_manager.cc:1397] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.031s user 0.019s sys 0.005s
I20260620 18:31:50.624600 11157 tablet_bootstrap.cc:492] T c865fa696725420ca2dec405247cf60c P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:31:50.625032 11157 ts_tablet_manager.cc:1397] T c865fa696725420ca2dec405247cf60c P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.043s user 0.007s sys 0.003s
I20260620 18:31:50.625073 10945 heartbeater.cc:503] Master 127.9.231.254:43203 was elected leader, sending a full tablet report...
I20260620 18:31:50.629881 11159 ts_tablet_manager.cc:1428] T c865fa696725420ca2dec405247cf60c P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.041s user 0.021s sys 0.000s
I20260620 18:31:50.630412 11159 tablet_bootstrap.cc:492] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
I20260620 18:31:50.631313 11158 raft_consensus.cc:348] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.631711 10408 catalog_manager.cc:2148] created HMS catalog entry for table [id=f01b90a2c4f74fb9aacaa125e3b4a9f2]
I20260620 18:31:50.631935 11158 raft_consensus.cc:374] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.632057 11158 raft_consensus.cc:729] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:31:50.632513 11159 tablet_bootstrap.cc:654] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:50.633103 11158 consensus_queue.cc:260] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.638087 11159 tablet_bootstrap.cc:492] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:31:50.638275 11159 ts_tablet_manager.cc:1397] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260620 18:31:50.639521 11159 raft_consensus.cc:348] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.639791 11159 raft_consensus.cc:374] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.639922 11159 raft_consensus.cc:729] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:31:50.639878 10143 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
/tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.253:44633
--webserver_interface=127.9.231.253
--webserver_port=37199
--builtin_ntp_servers=127.9.231.212:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260620 18:31:50.640221 11159 consensus_queue.cc:260] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.641000 11159 ts_tablet_manager.cc:1428] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:31:50.649747 11158 ts_tablet_manager.cc:1428] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.034s user 0.016s sys 0.003s
I20260620 18:31:50.649856 11157 raft_consensus.cc:348] T c865fa696725420ca2dec405247cf60c P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:50.650478 11157 raft_consensus.cc:374] T c865fa696725420ca2dec405247cf60c P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.650605 11157 raft_consensus.cc:729] T c865fa696725420ca2dec405247cf60c P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:31:50.651474 10797 heartbeater.cc:503] Master 127.9.231.254:43203 was elected leader, sending a full tablet report...
I20260620 18:31:50.651566 11157 consensus_queue.cc:260] T c865fa696725420ca2dec405247cf60c P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:50.653599 11158 tablet_bootstrap.cc:492] T c865fa696725420ca2dec405247cf60c P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:31:50.656652 11158 tablet_bootstrap.cc:654] T c865fa696725420ca2dec405247cf60c P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:50.662737 10730 tablet_service.cc:1467] Processing CreateTablet for tablet 4cdd0fd6fcb842239d0fb74758cd1290 (DEFAULT_TABLE table=default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.663268 10730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cdd0fd6fcb842239d0fb74758cd1290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:50.665226 11022 tablet_service.cc:1467] Processing CreateTablet for tablet 4cdd0fd6fcb842239d0fb74758cd1290 (DEFAULT_TABLE table=default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.665786 11022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cdd0fd6fcb842239d0fb74758cd1290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:50.667632 11158 tablet_bootstrap.cc:492] T c865fa696725420ca2dec405247cf60c P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:31:50.667955 11158 ts_tablet_manager.cc:1397] T c865fa696725420ca2dec405247cf60c P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.005s
I20260620 18:31:50.669780 11158 raft_consensus.cc:348] T c865fa696725420ca2dec405247cf60c P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:50.670253 11158 raft_consensus.cc:374] T c865fa696725420ca2dec405247cf60c P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.670454 11158 raft_consensus.cc:729] T c865fa696725420ca2dec405247cf60c P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:31:50.670926 11158 consensus_queue.cc:260] T c865fa696725420ca2dec405247cf60c P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:50.673432 11158 ts_tablet_manager.cc:1428] T c865fa696725420ca2dec405247cf60c P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260620 18:31:50.688516 10878 tablet_service.cc:1467] Processing CreateTablet for tablet 4cdd0fd6fcb842239d0fb74758cd1290 (DEFAULT_TABLE table=default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.689070 10878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cdd0fd6fcb842239d0fb74758cd1290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:50.690258 11157 ts_tablet_manager.cc:1428] T c865fa696725420ca2dec405247cf60c P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.065s user 0.034s sys 0.020s
I20260620 18:31:50.694419 11088 heartbeater.cc:503] Master 127.9.231.254:43203 was elected leader, sending a full tablet report...
I20260620 18:31:50.696041 11157 tablet_bootstrap.cc:492] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
I20260620 18:31:50.696632 11159 tablet_bootstrap.cc:492] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
I20260620 18:31:50.698779 11159 tablet_bootstrap.cc:654] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:50.704809 11158 tablet_bootstrap.cc:492] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:31:50.706806 11158 tablet_bootstrap.cc:654] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:50.707137 11157 tablet_bootstrap.cc:654] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:50.713572 11158 tablet_bootstrap.cc:492] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:31:50.713824 11158 ts_tablet_manager.cc:1397] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.001s
I20260620 18:31:50.715015 11158 raft_consensus.cc:348] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:50.715229 10408 master_service.cc:946] ReplaceTablet: received request to replace tablet c865fa696725420ca2dec405247cf60c from {username='slave'} at 127.0.0.1:36824
I20260620 18:31:50.715349 11158 raft_consensus.cc:374] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.715426 11158 raft_consensus.cc:729] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:31:50.715714 11158 consensus_queue.cc:260] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:50.716523 11158 ts_tablet_manager.cc:1428] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260620 18:31:50.718146 11159 tablet_bootstrap.cc:492] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:31:50.718360 11159 ts_tablet_manager.cc:1397] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.022s user 0.005s sys 0.005s
I20260620 18:31:50.719309 11159 raft_consensus.cc:348] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:50.719587 11159 raft_consensus.cc:374] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.719667 11159 raft_consensus.cc:729] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:31:50.719965 11159 consensus_queue.cc:260] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:50.720672 11159 ts_tablet_manager.cc:1428] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 18:31:50.724146 11157 tablet_bootstrap.cc:492] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:31:50.730273 11157 ts_tablet_manager.cc:1397] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.034s user 0.005s sys 0.000s
I20260620 18:31:50.731734 11157 raft_consensus.cc:348] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.732036 11157 raft_consensus.cc:374] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.732131 11157 raft_consensus.cc:729] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:31:50.732471 11157 consensus_queue.cc:260] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.733263 11157 ts_tablet_manager.cc:1428] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260620 18:31:50.733680 11157 tablet_bootstrap.cc:492] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
I20260620 18:31:50.735814 11157 tablet_bootstrap.cc:654] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:50.736768 10408 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Sending DeleteTablet for 3 replicas of tablet c865fa696725420ca2dec405247cf60c
I20260620 18:31:50.737743 10408 catalog_manager.cc:6565] ReplaceTablet: tablet c865fa696725420ca2dec405247cf60c deleted and replaced by tablet b7a80fba1822404a98459c47faef4680
I20260620 18:31:50.742801 10730 tablet_service.cc:1514] Processing DeleteTablet for tablet c865fa696725420ca2dec405247cf60c with delete_type TABLET_DATA_DELETED (replaced by tablet b7a80fba1822404a98459c47faef4680) from {username='slave'} at 127.0.0.1:34040
I20260620 18:31:50.743848 11022 tablet_service.cc:1514] Processing DeleteTablet for tablet c865fa696725420ca2dec405247cf60c with delete_type TABLET_DATA_DELETED (replaced by tablet b7a80fba1822404a98459c47faef4680) from {username='slave'} at 127.0.0.1:58206
I20260620 18:31:50.747150 10410 catalog_manager.cc:2148] created HMS catalog entry for table [id=4f2995259ed3447491bbea0b9afa3c74]
I20260620 18:31:50.754983 11164 raft_consensus.cc:482] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:50.755219 10878 tablet_service.cc:1514] Processing DeleteTablet for tablet c865fa696725420ca2dec405247cf60c with delete_type TABLET_DATA_DELETED (replaced by tablet b7a80fba1822404a98459c47faef4680) from {username='slave'} at 127.0.0.1:54440
I20260620 18:31:50.755497 11164 raft_consensus.cc:504] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.756561 11177 tablet_replica.cc:330] T c865fa696725420ca2dec405247cf60c P a0ae894b0758446aa6219aaa5267d1b4: stopping tablet replica
I20260620 18:31:50.757189 11177 raft_consensus.cc:2229] T c865fa696725420ca2dec405247cf60c P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260620 18:31:50.757702 11177 raft_consensus.cc:2258] T c865fa696725420ca2dec405247cf60c P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260620 18:31:50.759040 11021 tablet_service.cc:1467] Processing CreateTablet for tablet b7a80fba1822404a98459c47faef4680 (DEFAULT_TABLE table=default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.759583 11021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7a80fba1822404a98459c47faef4680. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:50.759505 10731 tablet_service.cc:1467] Processing CreateTablet for tablet b7a80fba1822404a98459c47faef4680 (DEFAULT_TABLE table=default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.759931 10731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7a80fba1822404a98459c47faef4680. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:50.763209 10410 master_service.cc:946] ReplaceTablet: received request to replace tablet 1d85fc3e284f48f080b205bfdcb55e6e from {username='slave'} at 127.0.0.1:36824
I20260620 18:31:50.764389 11167 raft_consensus.cc:482] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:50.764737 11167 raft_consensus.cc:504] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:50.767145 11167 leader_election.cc:288] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:31:50.770931 11177 ts_tablet_manager.cc:1905] T c865fa696725420ca2dec405247cf60c P a0ae894b0758446aa6219aaa5267d1b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:31:50.774850 11164 leader_election.cc:288] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:50.777392 11042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d85fc3e284f48f080b205bfdcb55e6e" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc" is_pre_election: true
I20260620 18:31:50.778115 11042 raft_consensus.cc:2454] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 0.
I20260620 18:31:50.779258 10833 leader_election.cc:302] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:31:50.779424 11177 ts_tablet_manager.cc:1918] T c865fa696725420ca2dec405247cf60c P a0ae894b0758446aa6219aaa5267d1b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260620 18:31:50.779646 11177 log.cc:1196] T c865fa696725420ca2dec405247cf60c P a0ae894b0758446aa6219aaa5267d1b4: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/wal/wals/c865fa696725420ca2dec405247cf60c
I20260620 18:31:50.779819 11164 raft_consensus.cc:2781] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:31:50.779989 11164 raft_consensus.cc:482] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:31:50.780112 11164 raft_consensus.cc:3037] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:50.780288 11177 ts_tablet_manager.cc:1939] T c865fa696725420ca2dec405247cf60c P a0ae894b0758446aa6219aaa5267d1b4: Deleting consensus metadata
I20260620 18:31:50.785161 11164 raft_consensus.cc:504] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.785997 11164 leader_election.cc:288] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Requested vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:50.787873 11158 tablet_bootstrap.cc:492] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:31:50.788316 11042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d85fc3e284f48f080b205bfdcb55e6e" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc"
I20260620 18:31:50.788627 11042 raft_consensus.cc:3037] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:50.789865 11158 tablet_bootstrap.cc:654] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:50.790545 10878 tablet_service.cc:1467] Processing CreateTablet for tablet b7a80fba1822404a98459c47faef4680 (DEFAULT_TABLE table=default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.791116 10878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7a80fba1822404a98459c47faef4680. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:50.793165 10410 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Sending DeleteTablet for 3 replicas of tablet 1d85fc3e284f48f080b205bfdcb55e6e
I20260620 18:31:50.794809 11021 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d85fc3e284f48f080b205bfdcb55e6e with delete_type TABLET_DATA_DELETED (replaced by tablet 1709fe2d11a24f52bd1812bc37e1714b) from {username='slave'} at 127.0.0.1:58206
I20260620 18:31:50.795153 10410 catalog_manager.cc:6565] ReplaceTablet: tablet 1d85fc3e284f48f080b205bfdcb55e6e deleted and replaced by tablet 1709fe2d11a24f52bd1812bc37e1714b
I20260620 18:31:50.796811 10403 catalog_manager.cc:4841] TS a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): tablet c865fa696725420ca2dec405247cf60c (table default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3]) successfully deleted
I20260620 18:31:50.797875 10879 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d85fc3e284f48f080b205bfdcb55e6e with delete_type TABLET_DATA_DELETED (replaced by tablet 1709fe2d11a24f52bd1812bc37e1714b) from {username='slave'} at 127.0.0.1:54440
I20260620 18:31:50.807863 11177 tablet_replica.cc:330] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4: stopping tablet replica
I20260620 18:31:50.808140 11177 raft_consensus.cc:2229] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:31:50.808403 11177 raft_consensus.cc:2258] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:31:50.808771 11159 tablet_bootstrap.cc:492] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
I20260620 18:31:50.810935 11159 tablet_bootstrap.cc:654] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
W20260620 18:31:50.811506 10799 tablet.cc:2369] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:31:50.812137 11021 tablet_service.cc:1467] Processing CreateTablet for tablet 8009d3fea30b4efa951e07d0bc2da1a5 (DEFAULT_TABLE table=default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.813007 11021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8009d3fea30b4efa951e07d0bc2da1a5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260620 18:31:50.813957 10947 tablet.cc:2369] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:31:50.820588 11042 raft_consensus.cc:2454] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
I20260620 18:31:50.821411 10833 leader_election.cc:302] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4; no voters:
W20260620 18:31:50.821717 10833 raft_consensus.cc:2670] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260620 18:31:50.826071 10410 master_service.cc:946] ReplaceTablet: received request to replace tablet c865fa696725420ca2dec405247cf60c from {username='slave'} at 127.0.0.1:36824
I20260620 18:31:50.827740 10410 master_service.cc:946] ReplaceTablet: received request to replace tablet 1d85fc3e284f48f080b205bfdcb55e6e from {username='slave'} at 127.0.0.1:36824
I20260620 18:31:50.828860 10410 master_service.cc:946] ReplaceTablet: received request to replace tablet 1d85fc3e284f48f080b205bfdcb55e6e from {username='slave'} at 127.0.0.1:36824
I20260620 18:31:50.829411 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d85fc3e284f48f080b205bfdcb55e6e" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433"
I20260620 18:31:50.829905 10750 raft_consensus.cc:3037] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:50.830432 11158 tablet_bootstrap.cc:492] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:31:50.830644 11158 ts_tablet_manager.cc:1397] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.043s user 0.017s sys 0.000s
I20260620 18:31:50.831655 11158 raft_consensus.cc:348] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.832018 11158 raft_consensus.cc:374] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.832152 11158 raft_consensus.cc:729] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:31:50.832392 10751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d85fc3e284f48f080b205bfdcb55e6e" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:31:50.833014 11177 ts_tablet_manager.cc:1905] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:31:50.835050 10750 raft_consensus.cc:2454] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
I20260620 18:31:50.837461 10878 tablet_service.cc:1467] Processing CreateTablet for tablet 8009d3fea30b4efa951e07d0bc2da1a5 (DEFAULT_TABLE table=default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.837989 10878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8009d3fea30b4efa951e07d0bc2da1a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:50.840727 11158 consensus_queue.cc:260] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.844717 11158 ts_tablet_manager.cc:1428] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.014s user 0.003s sys 0.000s
I20260620 18:31:50.847327 10410 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36824:
table { table_name: "default.Table_23d8d896ffe7406ca154449fd1a1eeab" } new_table_name: "default.Table_042b39ab8bd94475b7844eed01c35807" modify_external_catalogs: true
I20260620 18:31:50.848764 11159 tablet_bootstrap.cc:492] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:31:50.848963 11159 ts_tablet_manager.cc:1397] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.040s user 0.011s sys 0.005s
I20260620 18:31:50.849864 11159 raft_consensus.cc:348] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.850148 11159 raft_consensus.cc:374] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.850230 11159 raft_consensus.cc:729] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:31:50.851869 11021 tablet_service.cc:1467] Processing CreateTablet for tablet 1709fe2d11a24f52bd1812bc37e1714b (DEFAULT_TABLE table=default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.852638 11021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1709fe2d11a24f52bd1812bc37e1714b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:50.855376 11177 ts_tablet_manager.cc:1918] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260620 18:31:50.855603 11177 log.cc:1196] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/wal/wals/1d85fc3e284f48f080b205bfdcb55e6e
I20260620 18:31:50.856071 11177 ts_tablet_manager.cc:1939] T 1d85fc3e284f48f080b205bfdcb55e6e P a0ae894b0758446aa6219aaa5267d1b4: Deleting consensus metadata
I20260620 18:31:50.857658 11159 consensus_queue.cc:260] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.858515 11159 ts_tablet_manager.cc:1428] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260620 18:31:50.860107 11159 tablet_bootstrap.cc:492] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
I20260620 18:31:50.861394 10408 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36824:
name: "default.Table_16d5e29e7ed8458180884abae667bf9d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:50.862241 11159 tablet_bootstrap.cc:654] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:50.866223 10878 tablet_service.cc:1467] Processing CreateTablet for tablet 1709fe2d11a24f52bd1812bc37e1714b (DEFAULT_TABLE table=default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.866967 10878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1709fe2d11a24f52bd1812bc37e1714b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:50.874293 11159 tablet_bootstrap.cc:492] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:31:50.874577 11159 ts_tablet_manager.cc:1397] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260620 18:31:50.875711 11159 raft_consensus.cc:348] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.876127 11159 raft_consensus.cc:374] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.876231 11159 raft_consensus.cc:729] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:31:50.876427 11164 raft_consensus.cc:482] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:50.876567 11164 raft_consensus.cc:504] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:50.876613 11159 consensus_queue.cc:260] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.877415 11164 leader_election.cc:288] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:50.879299 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cdd0fd6fcb842239d0fb74758cd1290" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:31:50.879739 10750 raft_consensus.cc:2454] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 0.
I20260620 18:31:50.881029 11175 tablet_replica.cc:330] T c865fa696725420ca2dec405247cf60c P 7696506dcc9746d0aea9b1129a3e9433: stopping tablet replica
I20260620 18:31:50.881948 11175 raft_consensus.cc:2229] T c865fa696725420ca2dec405247cf60c P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260620 18:31:50.882457 11175 raft_consensus.cc:2258] T c865fa696725420ca2dec405247cf60c P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260620 18:31:50.864666 10403 catalog_manager.cc:4841] TS a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): tablet 1d85fc3e284f48f080b205bfdcb55e6e (table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]) successfully deleted
I20260620 18:31:50.883816 10832 leader_election.cc:302] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:31:50.884279 11164 raft_consensus.cc:2781] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:31:50.884426 11164 raft_consensus.cc:482] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:31:50.884532 11164 raft_consensus.cc:3037] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:50.886839 11176 tablet_replica.cc:330] T c865fa696725420ca2dec405247cf60c P 9a4ce671d3984e1a8592bc29d45612bc: stopping tablet replica
I20260620 18:31:50.887346 11176 raft_consensus.cc:2229] T c865fa696725420ca2dec405247cf60c P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Raft consensus shutting down.
I20260620 18:31:50.887833 11176 raft_consensus.cc:2258] T c865fa696725420ca2dec405247cf60c P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Raft consensus is shut down!
I20260620 18:31:50.888003 11164 raft_consensus.cc:504] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:50.888988 11164 leader_election.cc:288] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Requested vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:50.889961 11176 ts_tablet_manager.cc:1905] T c865fa696725420ca2dec405247cf60c P 9a4ce671d3984e1a8592bc29d45612bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:31:50.889961 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cdd0fd6fcb842239d0fb74758cd1290" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433"
I20260620 18:31:50.890352 10750 raft_consensus.cc:3037] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:50.890094 11041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cdd0fd6fcb842239d0fb74758cd1290" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc"
W20260620 18:31:50.891940 10833 leader_election.cc:341] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Illegal state: must be running to vote when last-logged opid is not known
I20260620 18:31:50.888835 11042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cdd0fd6fcb842239d0fb74758cd1290" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc" is_pre_election: true
W20260620 18:31:50.893134 10833 leader_election.cc:341] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Illegal state: must be running to vote when last-logged opid is not known
I20260620 18:31:50.893759 10750 raft_consensus.cc:2454] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
I20260620 18:31:50.894435 10832 leader_election.cc:302] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, a0ae894b0758446aa6219aaa5267d1b4; no voters: 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:31:50.894870 11164 raft_consensus.cc:2781] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:31:50.895061 11164 raft_consensus.cc:686] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 LEADER]: Becoming Leader. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Running, Role: LEADER
I20260620 18:31:50.895568 11164 consensus_queue.cc:237] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:50.902433 10730 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d85fc3e284f48f080b205bfdcb55e6e with delete_type TABLET_DATA_DELETED (replaced by tablet 1709fe2d11a24f52bd1812bc37e1714b) from {username='slave'} at 127.0.0.1:34040
I20260620 18:31:50.903043 10731 tablet_service.cc:1467] Processing CreateTablet for tablet 8009d3fea30b4efa951e07d0bc2da1a5 (DEFAULT_TABLE table=default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.903517 10731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8009d3fea30b4efa951e07d0bc2da1a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:50.903800 11175 ts_tablet_manager.cc:1905] T c865fa696725420ca2dec405247cf60c P 7696506dcc9746d0aea9b1129a3e9433: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:31:50.905565 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cdd0fd6fcb842239d0fb74758cd1290" candidate_uuid: "7696506dcc9746d0aea9b1129a3e9433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4" is_pre_election: true
I20260620 18:31:50.912317 10730 tablet_service.cc:1467] Processing CreateTablet for tablet 1709fe2d11a24f52bd1812bc37e1714b (DEFAULT_TABLE table=default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:50.912837 10730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1709fe2d11a24f52bd1812bc37e1714b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:50.914120 11158 tablet_bootstrap.cc:492] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:31:50.916183 11158 tablet_bootstrap.cc:654] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:50.916458 11159 ts_tablet_manager.cc:1428] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.042s user 0.009s sys 0.001s
I20260620 18:31:50.917066 11159 tablet_bootstrap.cc:492] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
I20260620 18:31:50.919085 11159 tablet_bootstrap.cc:654] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:50.921267 11042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cdd0fd6fcb842239d0fb74758cd1290" candidate_uuid: "7696506dcc9746d0aea9b1129a3e9433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc" is_pre_election: true
I20260620 18:31:50.921414 11175 ts_tablet_manager.cc:1918] T c865fa696725420ca2dec405247cf60c P 7696506dcc9746d0aea9b1129a3e9433: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260620 18:31:50.921609 11175 log.cc:1196] T c865fa696725420ca2dec405247cf60c P 7696506dcc9746d0aea9b1129a3e9433: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal/wals/c865fa696725420ca2dec405247cf60c
I20260620 18:31:50.922070 11175 ts_tablet_manager.cc:1939] T c865fa696725420ca2dec405247cf60c P 7696506dcc9746d0aea9b1129a3e9433: Deleting consensus metadata
W20260620 18:31:50.922266 10687 leader_election.cc:341] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Illegal state: must be running to vote when last-logged opid is not known
I20260620 18:31:50.922524 10687 leader_election.cc:302] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:31:50.923328 11167 raft_consensus.cc:2726] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:31:50.924348 11175 tablet_replica.cc:330] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433: stopping tablet replica
I20260620 18:31:50.924435 10402 catalog_manager.cc:4841] TS 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): tablet c865fa696725420ca2dec405247cf60c (table default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3]) successfully deleted
I20260620 18:31:50.924574 11175 raft_consensus.cc:2229] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:31:50.924741 11175 raft_consensus.cc:2258] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:31:50.925661 11175 ts_tablet_manager.cc:1905] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:31:50.933146 11175 ts_tablet_manager.cc:1918] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260620 18:31:50.933315 11175 log.cc:1196] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal/wals/1d85fc3e284f48f080b205bfdcb55e6e
I20260620 18:31:50.933679 11175 ts_tablet_manager.cc:1939] T 1d85fc3e284f48f080b205bfdcb55e6e P 7696506dcc9746d0aea9b1129a3e9433: Deleting consensus metadata
I20260620 18:31:50.933820 11158 tablet_bootstrap.cc:492] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:31:50.934310 11158 ts_tablet_manager.cc:1397] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.001s
I20260620 18:31:50.935238 11158 raft_consensus.cc:348] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.935391 10402 catalog_manager.cc:4841] TS 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): tablet 1d85fc3e284f48f080b205bfdcb55e6e (table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]) successfully deleted
I20260620 18:31:50.935683 11158 raft_consensus.cc:374] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.935765 11158 raft_consensus.cc:729] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
W20260620 18:31:50.936328 11090 tablet.cc:2369] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:31:50.937057 11157 tablet_bootstrap.cc:492] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:31:50.937237 11157 ts_tablet_manager.cc:1397] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.204s user 0.034s sys 0.040s
I20260620 18:31:50.937058 11158 consensus_queue.cc:260] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.938177 11158 ts_tablet_manager.cc:1428] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260620 18:31:50.938073 11157 raft_consensus.cc:348] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:50.938323 11157 raft_consensus.cc:374] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.938396 11157 raft_consensus.cc:729] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:31:50.939127 11157 consensus_queue.cc:260] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:50.939719 11157 ts_tablet_manager.cc:1428] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260620 18:31:50.940073 11157 tablet_bootstrap.cc:492] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
I20260620 18:31:50.940963 11159 tablet_bootstrap.cc:492] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:31:50.941183 11159 ts_tablet_manager.cc:1397] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.001s
I20260620 18:31:50.941977 11157 tablet_bootstrap.cc:654] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:50.942569 11159 raft_consensus.cc:348] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.942906 11159 raft_consensus.cc:374] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.943044 11159 raft_consensus.cc:729] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:31:50.943425 11159 consensus_queue.cc:260] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.944339 11158 tablet_bootstrap.cc:492] T 1709fe2d11a24f52bd1812bc37e1714b P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:31:50.946669 11158 tablet_bootstrap.cc:654] T 1709fe2d11a24f52bd1812bc37e1714b P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:50.948685 11159 ts_tablet_manager.cc:1428] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260620 18:31:50.949841 11158 tablet_bootstrap.cc:492] T 1709fe2d11a24f52bd1812bc37e1714b P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:31:50.950052 11158 ts_tablet_manager.cc:1397] T 1709fe2d11a24f52bd1812bc37e1714b P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260620 18:31:50.950883 11158 raft_consensus.cc:348] T 1709fe2d11a24f52bd1812bc37e1714b P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.951215 11158 raft_consensus.cc:374] T 1709fe2d11a24f52bd1812bc37e1714b P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.951380 11158 raft_consensus.cc:729] T 1709fe2d11a24f52bd1812bc37e1714b P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:31:50.951704 11158 consensus_queue.cc:260] T 1709fe2d11a24f52bd1812bc37e1714b P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.952445 11158 ts_tablet_manager.cc:1428] T 1709fe2d11a24f52bd1812bc37e1714b P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 18:31:50.958061 11176 ts_tablet_manager.cc:1918] T c865fa696725420ca2dec405247cf60c P 9a4ce671d3984e1a8592bc29d45612bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260620 18:31:50.958302 11176 log.cc:1196] T c865fa696725420ca2dec405247cf60c P 9a4ce671d3984e1a8592bc29d45612bc: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal/wals/c865fa696725420ca2dec405247cf60c
I20260620 18:31:50.958966 11176 ts_tablet_manager.cc:1939] T c865fa696725420ca2dec405247cf60c P 9a4ce671d3984e1a8592bc29d45612bc: Deleting consensus metadata
I20260620 18:31:50.961218 10403 catalog_manager.cc:4841] TS 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): tablet c865fa696725420ca2dec405247cf60c (table default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3]) successfully deleted
I20260620 18:31:50.963204 11176 tablet_replica.cc:330] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc: stopping tablet replica
I20260620 18:31:50.963608 11176 raft_consensus.cc:2229] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:31:50.964504 11176 raft_consensus.cc:2258] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:31:50.968838 11176 ts_tablet_manager.cc:1905] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:31:50.973783 11157 tablet_bootstrap.cc:492] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:31:50.974013 11157 ts_tablet_manager.cc:1397] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.034s user 0.006s sys 0.010s
I20260620 18:31:50.975006 11157 raft_consensus.cc:348] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.975332 11157 raft_consensus.cc:374] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.975448 11157 raft_consensus.cc:729] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:31:50.975731 11157 consensus_queue.cc:260] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.977219 11157 ts_tablet_manager.cc:1428] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260620 18:31:50.977440 11176 ts_tablet_manager.cc:1918] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260620 18:31:50.977589 11176 log.cc:1196] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal/wals/1d85fc3e284f48f080b205bfdcb55e6e
I20260620 18:31:50.978003 11176 ts_tablet_manager.cc:1939] T 1d85fc3e284f48f080b205bfdcb55e6e P 9a4ce671d3984e1a8592bc29d45612bc: Deleting consensus metadata
I20260620 18:31:50.977588 11157 tablet_bootstrap.cc:492] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
I20260620 18:31:50.979333 10403 catalog_manager.cc:4841] TS 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): tablet 1d85fc3e284f48f080b205bfdcb55e6e (table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]) successfully deleted
I20260620 18:31:50.981101 11157 tablet_bootstrap.cc:654] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:50.983814 11157 tablet_bootstrap.cc:492] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:31:50.984074 11157 ts_tablet_manager.cc:1397] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260620 18:31:50.984910 11157 raft_consensus.cc:348] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.985281 11157 raft_consensus.cc:374] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.985406 11157 raft_consensus.cc:729] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:31:50.985669 11157 consensus_queue.cc:260] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.987365 11157 ts_tablet_manager.cc:1428] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260620 18:31:50.987718 11157 tablet_bootstrap.cc:492] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
I20260620 18:31:50.989996 11157 tablet_bootstrap.cc:654] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:50.994553 11157 tablet_bootstrap.cc:492] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:31:50.994788 11157 ts_tablet_manager.cc:1397] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260620 18:31:50.995798 11157 raft_consensus.cc:348] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.996073 11157 raft_consensus.cc:374] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:50.996157 11157 raft_consensus.cc:729] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:31:50.996906 11157 consensus_queue.cc:260] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:50.997706 11157 ts_tablet_manager.cc:1428] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260620 18:31:51.000523 11183 raft_consensus.cc:482] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:51.000818 11183 raft_consensus.cc:504] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.001777 11183 leader_election.cc:288] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:51.002390 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1709fe2d11a24f52bd1812bc37e1714b" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:31:51.002668 10750 raft_consensus.cc:2454] T 1709fe2d11a24f52bd1812bc37e1714b P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 0.
I20260620 18:31:51.003473 10832 leader_election.cc:302] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:31:51.003813 11042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1709fe2d11a24f52bd1812bc37e1714b" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc" is_pre_election: true
I20260620 18:31:51.004117 11042 raft_consensus.cc:2454] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 0.
I20260620 18:31:51.004854 11183 raft_consensus.cc:2781] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:31:51.004988 11183 raft_consensus.cc:482] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:31:51.005069 11183 raft_consensus.cc:3037] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:51.009298 11183 raft_consensus.cc:504] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.010756 11183 leader_election.cc:288] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Requested vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:51.010934 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1709fe2d11a24f52bd1812bc37e1714b" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433"
I20260620 18:31:51.010983 11042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1709fe2d11a24f52bd1812bc37e1714b" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc"
I20260620 18:31:51.011198 10750 raft_consensus.cc:3037] T 1709fe2d11a24f52bd1812bc37e1714b P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:51.011341 11042 raft_consensus.cc:3037] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:51.013517 10750 raft_consensus.cc:2454] T 1709fe2d11a24f52bd1812bc37e1714b P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
I20260620 18:31:51.014129 10832 leader_election.cc:302] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:31:51.014487 11183 raft_consensus.cc:2781] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:31:51.014791 11183 raft_consensus.cc:686] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [term 1 LEADER]: Becoming Leader. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Running, Role: LEADER
I20260620 18:31:51.015273 11183 consensus_queue.cc:237] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.017112 11042 raft_consensus.cc:2454] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
I20260620 18:31:51.022903 10412 catalog_manager.cc:2148] created HMS catalog entry for table [id=2a7c8da0fac442988ce595b8b74d4047]
W20260620 18:31:51.033252 10407 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f27c96a5d11944f6a75123e42b5b2ce5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260620 18:31:51.034528 10411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_88afa4cb37564293a220876026ee1931 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260620 18:31:51.035622 10409 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_08795a50aec84f85bc35ae8025674185 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260620 18:31:51.036540 10416 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_aa7e29fa2b8e4974a8ece0e46aba222a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260620 18:31:51.037443 10415 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1ba08641833a48838fae05c1f72333e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
F20260620 18:31:51.047479 10614 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_269d7d1648864d4e8b29b944b7115e6a
*** Check failure stack trace: ***
@ 0x7f5bbdb459ad google::LogMessage::Fail() at ??:0
@ 0x7f5bbdb4888a google::LogMessage::SendToLog() at ??:0
I20260620 18:31:51.053378 11021 tablet_service.cc:1467] Processing CreateTablet for tablet 4fb0f7d77bac4b11bc0aa4486b39aa97 (DEFAULT_TABLE table=default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:51.053750 10878 tablet_service.cc:1467] Processing CreateTablet for tablet 4fb0f7d77bac4b11bc0aa4486b39aa97 (DEFAULT_TABLE table=default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:51.053956 11021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fb0f7d77bac4b11bc0aa4486b39aa97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:51.054080 10730 tablet_service.cc:1467] Processing CreateTablet for tablet 4fb0f7d77bac4b11bc0aa4486b39aa97 (DEFAULT_TABLE table=default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:51.054322 10878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fb0f7d77bac4b11bc0aa4486b39aa97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:51.054493 10730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fb0f7d77bac4b11bc0aa4486b39aa97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:51.061583 11159 tablet_bootstrap.cc:492] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
I20260620 18:31:51.063678 11159 tablet_bootstrap.cc:654] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:51.067994 10412 catalog_manager.cc:5495] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194). New cstate: current_term: 1 leader_uuid: "a0ae894b0758446aa6219aaa5267d1b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } health_report { overall_health: UNKNOWN } } }
I20260620 18:31:51.069332 10412 catalog_manager.cc:5495] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194). New cstate: current_term: 1 leader_uuid: "a0ae894b0758446aa6219aaa5267d1b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } health_report { overall_health: HEALTHY } } }
@ 0x7f5bbdb454de google::LogMessage::Flush() at ??:0
@ 0x7f5bbdb456d1 google::LogMessage::~LogMessage() at ??:0
I20260620 18:31:51.076988 11169 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:51.077230 11169 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.080499 11157 tablet_bootstrap.cc:492] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
I20260620 18:31:51.080616 11169 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:31:51.082458 11157 tablet_bootstrap.cc:654] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:51.095664 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8009d3fea30b4efa951e07d0bc2da1a5" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:31:51.096336 10750 raft_consensus.cc:2454] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 0.
I20260620 18:31:51.097054 10977 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc; no voters:
I20260620 18:31:51.097527 11169 raft_consensus.cc:2781] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:31:51.097640 11169 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:31:51.097715 11169 raft_consensus.cc:3037] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
@ 0x7f5bd36e3ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260620 18:31:51.104897 11169 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.114928 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8009d3fea30b4efa951e07d0bc2da1a5" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433"
I20260620 18:31:51.115262 10750 raft_consensus.cc:3037] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:51.118142 10750 raft_consensus.cc:2454] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 1.
I20260620 18:31:51.118769 10977 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc; no voters:
I20260620 18:31:51.125780 11169 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 election: Requested vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:31:51.126045 11169 raft_consensus.cc:2781] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:31:51.126201 11169 raft_consensus.cc:686] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 LEADER]: Becoming Leader. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Running, Role: LEADER
I20260620 18:31:51.126654 11169 consensus_queue.cc:237] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.132814 11158 tablet_bootstrap.cc:492] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:31:51.134778 11158 tablet_bootstrap.cc:654] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:51.138885 11167 raft_consensus.cc:482] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:51.139087 11167 raft_consensus.cc:504] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.139936 11167 leader_election.cc:288] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:51.140874 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a80fba1822404a98459c47faef4680" candidate_uuid: "7696506dcc9746d0aea9b1129a3e9433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4" is_pre_election: true
I20260620 18:31:51.141281 10899 raft_consensus.cc:2454] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7696506dcc9746d0aea9b1129a3e9433 in term 0.
I20260620 18:31:51.142059 10687 leader_election.cc:302] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:31:51.142449 11167 raft_consensus.cc:2781] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:31:51.142599 11167 raft_consensus.cc:482] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:31:51.142699 11167 raft_consensus.cc:3037] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:51.146934 11167 raft_consensus.cc:504] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.147886 10407 catalog_manager.cc:2148] created HMS catalog entry for table [id=3b76eebe1edc48a0923cb81e36eb2770]
I20260620 18:31:51.148278 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a80fba1822404a98459c47faef4680" candidate_uuid: "7696506dcc9746d0aea9b1129a3e9433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4"
I20260620 18:31:51.148527 10899 raft_consensus.cc:3037] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:51.151643 10899 raft_consensus.cc:2454] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7696506dcc9746d0aea9b1129a3e9433 in term 1.
I20260620 18:31:51.152827 11167 leader_election.cc:288] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 election: Requested vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:51.153220 10687 leader_election.cc:302] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:31:51.157979 10407 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:36824 (request call id 11) took 1087 ms. Trace:
I20260620 18:31:51.158262 11167 raft_consensus.cc:2781] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:31:51.158174 10407 rpcz_store.cc:276] 0620 18:31:50.070877 (+ 0us) service_pool.cc:170] Inserting onto call queue
0620 18:31:50.074718 (+ 3841us) service_pool.cc:227] Handling call
0620 18:31:51.033433 (+958715us) catalog_manager.cc:2075] Acquired catalog manager lock
0620 18:31:51.034002 (+ 569us) catalog_manager.cc:2121] Created new table and tablet info
0620 18:31:51.147824 (+113822us) catalog_manager.cc:2147] Created new table in HMS catalog
0620 18:31:51.157685 (+ 9861us) catalog_manager.cc:2181] Wrote table and tablets to system table
0620 18:31:51.157816 (+ 131us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0620 18:31:51.157935 (+ 119us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0620 18:31:51.148786 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0620 18:31:51.148923 (+ 137us) write_op.cc:432] Acquiring schema lock in shared mode
0620 18:31:51.148932 (+ 9us) write_op.cc:435] Acquired schema lock
0620 18:31:51.148935 (+ 3us) tablet.cc:660] Decoding operations
0620 18:31:51.149036 (+ 101us) write_op.cc:620] Acquiring the partition lock for write op
0620 18:31:51.149051 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0620 18:31:51.149056 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0620 18:31:51.149148 (+ 92us) tablet.cc:699] Row locks acquired
0620 18:31:51.149150 (+ 2us) write_op.cc:260] PREPARE: finished
0620 18:31:51.149194 (+ 44us) write_op.cc:270] Start()
0620 18:31:51.149231 (+ 37us) write_op.cc:276] Timestamp: P: 1781980311149173 usec, L: 0
0620 18:31:51.149235 (+ 4us) op_driver.cc:331] REPLICATION: starting
0620 18:31:51.149490 (+ 255us) log.cc:842] Serialized 432 byte log entry
0620 18:31:51.156654 (+ 7164us) op_driver.cc:444] REPLICATION: finished
0620 18:31:51.156994 (+ 340us) write_op.cc:301] APPLY: starting
0620 18:31:51.157020 (+ 26us) tablet.cc:1365] starting BulkCheckPresence
0620 18:31:51.157098 (+ 78us) tablet.cc:1368] finished BulkCheckPresence
0620 18:31:51.157136 (+ 38us) tablet.cc:1370] starting ApplyRowOperation cycle
0620 18:31:51.157249 (+ 113us) tablet.cc:1381] finished ApplyRowOperation cycle
0620 18:31:51.157287 (+ 38us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0620 18:31:51.157293 (+ 6us) write_op.cc:312] APPLY: finished
0620 18:31:51.157383 (+ 90us) log.cc:842] Serialized 28 byte log entry
0620 18:31:51.157466 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0620 18:31:51.157490 (+ 24us) write_op.cc:454] Released schema lock
0620 18:31:51.157497 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0620 18:31:51.157558 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0620 18:31:51.157561 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":13063,"Hive Metastore.run_cpu_time_us":373,"Hive Metastore.run_wall_time_us":100207,"child_traces":[["op",{"apply.queue_time_us":268,"apply.run_cpu_time_us":632,"apply.run_wall_time_us":631,"num_ops":2,"prepare.queue_time_us":69,"prepare.run_cpu_time_us":850,"prepare.run_wall_time_us":850,"raft.queue_time_us":448,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":251,"replication_time_us":7381}]]}
I20260620 18:31:51.159740 10407 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36824:
name: "default.Table_e001ba7ef7b44f09b6262dc3d55d0df0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f5bd36e2901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260620 18:31:51.174479 10878 tablet_service.cc:1467] Processing CreateTablet for tablet 5151602cea1443a9a56935c28db34beb (DEFAULT_TABLE table=default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:51.174728 10731 tablet_service.cc:1467] Processing CreateTablet for tablet 5151602cea1443a9a56935c28db34beb (DEFAULT_TABLE table=default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:51.175169 10878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5151602cea1443a9a56935c28db34beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:51.175249 10731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5151602cea1443a9a56935c28db34beb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260620 18:31:51.186348 10402 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 11 similar messages]
@ 0x7f5bd36e966c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260620 18:31:51.200860 10412 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36824:
table { table_name: "default.Table_be97363936284e2faa8400049925a98c" } new_table_name: "default.Table_4e545c1bf96b40de8e03778dfe0e4861" modify_external_catalogs: true
W20260620 18:31:51.211565 10977 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.231.193:41961, user_credentials={real_user=slave}} blocked reactor thread for 92790.3us
I20260620 18:31:51.212468 11157 tablet_bootstrap.cc:492] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:31:51.212654 11157 ts_tablet_manager.cc:1397] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.132s user 0.010s sys 0.015s
I20260620 18:31:51.212597 11167 raft_consensus.cc:686] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 LEADER]: Becoming Leader. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Running, Role: LEADER
I20260620 18:31:51.213117 11167 consensus_queue.cc:237] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.213619 11157 raft_consensus.cc:348] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.213871 11157 raft_consensus.cc:374] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:51.213961 11157 raft_consensus.cc:729] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:31:51.214221 11157 consensus_queue.cc:260] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.214864 11157 ts_tablet_manager.cc:1428] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:31:51.215620 11021 tablet_service.cc:1467] Processing CreateTablet for tablet 5151602cea1443a9a56935c28db34beb (DEFAULT_TABLE table=default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:51.215893 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8009d3fea30b4efa951e07d0bc2da1a5" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4" is_pre_election: true
I20260620 18:31:51.216075 11021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5151602cea1443a9a56935c28db34beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:51.216183 10899 raft_consensus.cc:2454] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 0.
I20260620 18:31:51.217049 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8009d3fea30b4efa951e07d0bc2da1a5" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4"
I20260620 18:31:51.217247 10899 raft_consensus.cc:3037] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:51.220172 10899 raft_consensus.cc:2454] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 1.
I20260620 18:31:51.220739 11169 raft_consensus.cc:482] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:51.220907 11169 raft_consensus.cc:504] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.221671 11169 leader_election.cc:288] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:31:51.225137 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a80fba1822404a98459c47faef4680" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4" is_pre_election: true
I20260620 18:31:51.225471 10899 raft_consensus.cc:2379] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a4ce671d3984e1a8592bc29d45612bc in current term 1: Already voted for candidate 7696506dcc9746d0aea9b1129a3e9433 in this term.
I20260620 18:31:51.226563 11042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a80fba1822404a98459c47faef4680" candidate_uuid: "7696506dcc9746d0aea9b1129a3e9433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc" is_pre_election: true
I20260620 18:31:51.226799 11042 raft_consensus.cc:2454] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7696506dcc9746d0aea9b1129a3e9433 in term 0.
I20260620 18:31:51.228981 11042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a80fba1822404a98459c47faef4680" candidate_uuid: "7696506dcc9746d0aea9b1129a3e9433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc"
I20260620 18:31:51.229197 11042 raft_consensus.cc:3037] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:51.232545 11042 raft_consensus.cc:2454] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7696506dcc9746d0aea9b1129a3e9433 in term 1.
I20260620 18:31:51.233551 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a80fba1822404a98459c47faef4680" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:31:51.233963 10411 catalog_manager.cc:2148] created HMS catalog entry for table [id=df30a4bddc9b4511989d6c37feb16d59]
I20260620 18:31:51.234087 11159 tablet_bootstrap.cc:492] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:31:51.234333 11159 ts_tablet_manager.cc:1397] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.173s user 0.042s sys 0.045s
I20260620 18:31:51.235291 11159 raft_consensus.cc:348] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.235574 11159 raft_consensus.cc:374] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:51.235648 11159 raft_consensus.cc:729] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:31:51.237125 11157 tablet_bootstrap.cc:492] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
I20260620 18:31:51.239265 11159 consensus_queue.cc:260] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.240988 10411 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:36824 (request call id 7) took 1170 ms. Trace:
I20260620 18:31:51.241144 10411 rpcz_store.cc:276] 0620 18:31:50.070286 (+ 0us) service_pool.cc:170] Inserting onto call queue
0620 18:31:50.074497 (+ 4211us) service_pool.cc:227] Handling call
0620 18:31:51.034627 (+960130us) catalog_manager.cc:2075] Acquired catalog manager lock
0620 18:31:51.035176 (+ 549us) catalog_manager.cc:2121] Created new table and tablet info
0620 18:31:51.233904 (+198728us) catalog_manager.cc:2147] Created new table in HMS catalog
0620 18:31:51.240678 (+ 6774us) catalog_manager.cc:2181] Wrote table and tablets to system table
0620 18:31:51.240814 (+ 136us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0620 18:31:51.240924 (+ 110us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0620 18:31:51.234802 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0620 18:31:51.234930 (+ 128us) write_op.cc:432] Acquiring schema lock in shared mode
0620 18:31:51.234940 (+ 10us) write_op.cc:435] Acquired schema lock
0620 18:31:51.234943 (+ 3us) tablet.cc:660] Decoding operations
0620 18:31:51.235045 (+ 102us) write_op.cc:620] Acquiring the partition lock for write op
0620 18:31:51.235061 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0620 18:31:51.235066 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0620 18:31:51.235163 (+ 97us) tablet.cc:699] Row locks acquired
0620 18:31:51.235166 (+ 3us) write_op.cc:260] PREPARE: finished
0620 18:31:51.235206 (+ 40us) write_op.cc:270] Start()
0620 18:31:51.235256 (+ 50us) write_op.cc:276] Timestamp: P: 1781980311235187 usec, L: 0
0620 18:31:51.235259 (+ 3us) op_driver.cc:331] REPLICATION: starting
0620 18:31:51.235458 (+ 199us) log.cc:842] Serialized 432 byte log entry
0620 18:31:51.239670 (+ 4212us) op_driver.cc:444] REPLICATION: finished
0620 18:31:51.239840 (+ 170us) write_op.cc:301] APPLY: starting
0620 18:31:51.239857 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
0620 18:31:51.239904 (+ 47us) tablet.cc:1368] finished BulkCheckPresence
0620 18:31:51.239931 (+ 27us) tablet.cc:1370] starting ApplyRowOperation cycle
0620 18:31:51.240035 (+ 104us) tablet.cc:1381] finished ApplyRowOperation cycle
0620 18:31:51.240073 (+ 38us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0620 18:31:51.240079 (+ 6us) write_op.cc:312] APPLY: finished
0620 18:31:51.240163 (+ 84us) log.cc:842] Serialized 28 byte log entry
0620 18:31:51.240233 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0620 18:31:51.240280 (+ 47us) write_op.cc:454] Released schema lock
0620 18:31:51.240288 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0620 18:31:51.240344 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0620 18:31:51.240348 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":112196,"Hive Metastore.run_cpu_time_us":369,"Hive Metastore.run_wall_time_us":86035,"child_traces":[["op",{"apply.queue_time_us":116,"apply.run_cpu_time_us":563,"apply.run_wall_time_us":563,"num_ops":2,"prepare.queue_time_us":78,"prepare.run_cpu_time_us":782,"prepare.run_wall_time_us":782,"raft.queue_time_us":1239,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":254,"replication_time_us":4384}]]}
I20260620 18:31:51.242527 11159 ts_tablet_manager.cc:1428] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.008s user 0.002s sys 0.001s
I20260620 18:31:51.243119 10411 catalog_manager.cc:5495] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195). New cstate: current_term: 1 leader_uuid: "9a4ce671d3984e1a8592bc29d45612bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } health_report { overall_health: HEALTHY } } }
@ 0x7f5bd36e94e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260620 18:31:51.249397 11159 tablet_bootstrap.cc:492] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
I20260620 18:31:51.251232 11158 tablet_bootstrap.cc:492] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:31:51.251477 11158 ts_tablet_manager.cc:1397] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.119s user 0.001s sys 0.011s
I20260620 18:31:51.251689 11159 tablet_bootstrap.cc:654] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:51.252473 11158 raft_consensus.cc:348] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.238915 11157 tablet_bootstrap.cc:654] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:51.245291 10977 leader_election.cc:302] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a4ce671d3984e1a8592bc29d45612bc; no voters: 7696506dcc9746d0aea9b1129a3e9433, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:31:51.262012 11196 raft_consensus.cc:2726] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:31:51.264242 11159 tablet_bootstrap.cc:492] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:31:51.264456 11159 ts_tablet_manager.cc:1397] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260620 18:31:51.264853 11158 raft_consensus.cc:374] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:51.264994 11158 raft_consensus.cc:729] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:31:51.265261 10878 tablet_service.cc:1467] Processing CreateTablet for tablet 654f8832c4be4d9ea5d23b446696a63f (DEFAULT_TABLE table=default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:51.265725 10878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 654f8832c4be4d9ea5d23b446696a63f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:51.265643 11159 raft_consensus.cc:348] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.265887 11159 raft_consensus.cc:374] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:51.265975 11159 raft_consensus.cc:729] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:31:51.266219 11159 consensus_queue.cc:260] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.267204 10411 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36824:
name: "default.Table_5854ef7ddc554da59e75ed6ff18718ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:51.268898 11159 ts_tablet_manager.cc:1428] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260620 18:31:51.270117 10731 tablet_service.cc:1467] Processing CreateTablet for tablet 654f8832c4be4d9ea5d23b446696a63f (DEFAULT_TABLE table=default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:51.270627 10731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 654f8832c4be4d9ea5d23b446696a63f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:51.272083 11021 tablet_service.cc:1467] Processing CreateTablet for tablet 654f8832c4be4d9ea5d23b446696a63f (DEFAULT_TABLE table=default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:51.272601 11159 tablet_bootstrap.cc:492] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
I20260620 18:31:51.272809 11021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 654f8832c4be4d9ea5d23b446696a63f. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f5bc076feaf kudu::Thread::SuperviseThread() at ??:0
I20260620 18:31:51.274346 11159 tablet_bootstrap.cc:654] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:51.284737 11158 consensus_queue.cc:260] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.285440 11157 tablet_bootstrap.cc:492] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:31:51.285471 11158 ts_tablet_manager.cc:1428] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.034s user 0.001s sys 0.003s
I20260620 18:31:51.285614 11157 ts_tablet_manager.cc:1397] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.049s user 0.002s sys 0.001s
I20260620 18:31:51.286561 11157 raft_consensus.cc:348] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.286835 11157 raft_consensus.cc:374] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:51.286937 11157 raft_consensus.cc:729] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:31:51.287209 11157 consensus_queue.cc:260] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.289381 11157 ts_tablet_manager.cc:1428] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260620 18:31:51.300455 11158 tablet_bootstrap.cc:492] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
@ 0x7f5bc3a496db start_thread at ??:0
I20260620 18:31:51.303419 11157 tablet_bootstrap.cc:492] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
I20260620 18:31:51.305546 11157 tablet_bootstrap.cc:654] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:51.310389 11183 raft_consensus.cc:482] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:51.310711 11183 raft_consensus.cc:504] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.311713 11183 leader_election.cc:288] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
@ 0x7f5bbc9df61f clone at ??:0
I20260620 18:31:51.313411 11042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fb0f7d77bac4b11bc0aa4486b39aa97" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc" is_pre_election: true
I20260620 18:31:51.313818 11042 raft_consensus.cc:2454] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 0.
I20260620 18:31:51.314330 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fb0f7d77bac4b11bc0aa4486b39aa97" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:31:51.314467 10833 leader_election.cc:302] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:31:51.314745 10750 raft_consensus.cc:2454] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 0.
I20260620 18:31:51.314893 11183 raft_consensus.cc:2781] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:31:51.315024 11183 raft_consensus.cc:482] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:31:51.315153 11183 raft_consensus.cc:3037] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:51.318516 11183 raft_consensus.cc:504] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.319870 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fb0f7d77bac4b11bc0aa4486b39aa97" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433"
I20260620 18:31:51.320145 11157 tablet_bootstrap.cc:492] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:31:51.320078 11042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fb0f7d77bac4b11bc0aa4486b39aa97" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc"
I20260620 18:31:51.320204 10750 raft_consensus.cc:3037] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:51.320425 11042 raft_consensus.cc:3037] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:51.320755 11158 tablet_bootstrap.cc:654] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:51.324002 10750 raft_consensus.cc:2454] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
I20260620 18:31:51.324680 10832 leader_election.cc:302] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:31:51.325028 11183 leader_election.cc:288] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Requested vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:51.325055 11164 raft_consensus.cc:2781] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:31:51.324389 11042 raft_consensus.cc:2454] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
I20260620 18:31:51.325297 11159 tablet_bootstrap.cc:492] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:31:51.325351 11164 raft_consensus.cc:686] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 LEADER]: Becoming Leader. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Running, Role: LEADER
I20260620 18:31:51.325488 11159 ts_tablet_manager.cc:1397] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.053s user 0.016s sys 0.008s
I20260620 18:31:51.325846 11157 ts_tablet_manager.cc:1397] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.000s
I20260620 18:31:51.325783 11164 consensus_queue.cc:237] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.326411 11159 raft_consensus.cc:348] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.326717 11159 raft_consensus.cc:374] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:51.326805 11159 raft_consensus.cc:729] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:31:51.327062 11159 consensus_queue.cc:260] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.326676 11157 raft_consensus.cc:348] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.330857 11157 raft_consensus.cc:374] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:51.330968 11157 raft_consensus.cc:729] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:31:51.331288 11157 consensus_queue.cc:260] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.335476 11157 ts_tablet_manager.cc:1428] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260620 18:31:51.345443 11159 ts_tablet_manager.cc:1428] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.020s user 0.002s sys 0.006s
I20260620 18:31:51.347038 11158 tablet_bootstrap.cc:492] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:31:51.347213 11158 ts_tablet_manager.cc:1397] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.047s user 0.006s sys 0.003s
I20260620 18:31:51.348133 11158 raft_consensus.cc:348] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.348424 11158 raft_consensus.cc:374] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:51.348522 11158 raft_consensus.cc:729] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:31:51.348802 11158 consensus_queue.cc:260] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.351181 11158 ts_tablet_manager.cc:1428] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260620 18:31:51.352430 11158 tablet_bootstrap.cc:492] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:31:51.354274 11158 tablet_bootstrap.cc:654] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:51.377308 11158 tablet_bootstrap.cc:492] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:31:51.377519 11158 ts_tablet_manager.cc:1397] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.006s
I20260620 18:31:51.378257 11158 raft_consensus.cc:348] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.378551 11158 raft_consensus.cc:374] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:51.378623 11158 raft_consensus.cc:729] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:31:51.378974 11158 consensus_queue.cc:260] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.380482 11158 ts_tablet_manager.cc:1428] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260620 18:31:51.467427 11196 raft_consensus.cc:482] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:51.467672 11196 raft_consensus.cc:504] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.468612 11196 leader_election.cc:288] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:31:51.469604 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5151602cea1443a9a56935c28db34beb" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4" is_pre_election: true
I20260620 18:31:51.469875 10899 raft_consensus.cc:2454] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 0.
I20260620 18:31:51.470634 10978 leader_election.cc:302] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:31:51.470749 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5151602cea1443a9a56935c28db34beb" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:31:51.470994 10750 raft_consensus.cc:2454] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 0.
I20260620 18:31:51.471228 11196 raft_consensus.cc:2781] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:31:51.471328 11196 raft_consensus.cc:482] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:31:51.471441 11196 raft_consensus.cc:3037] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:51.474555 11196 raft_consensus.cc:504] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.475786 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5151602cea1443a9a56935c28db34beb" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433"
I20260620 18:31:51.476050 10750 raft_consensus.cc:3037] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:51.483757 10750 raft_consensus.cc:2454] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 1.
I20260620 18:31:51.484534 10977 leader_election.cc:302] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc; no voters:
I20260620 18:31:51.485065 11169 raft_consensus.cc:2781] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:31:51.485860 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5151602cea1443a9a56935c28db34beb" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4"
I20260620 18:31:51.486146 10899 raft_consensus.cc:3037] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:51.493973 11169 raft_consensus.cc:686] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [term 1 LEADER]: Becoming Leader. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Running, Role: LEADER
I20260620 18:31:51.494446 11169 consensus_queue.cc:237] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.498011 10899 raft_consensus.cc:2454] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 1.
I20260620 18:31:51.499015 11196 leader_election.cc:288] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 election: Requested vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:31:51.525334 11169 raft_consensus.cc:482] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:51.525646 11169 raft_consensus.cc:504] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:51.526515 11169 leader_election.cc:288] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:31:51.528236 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cdd0fd6fcb842239d0fb74758cd1290" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:31:51.528695 10750 raft_consensus.cc:2379] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a4ce671d3984e1a8592bc29d45612bc in current term 1: Already voted for candidate a0ae894b0758446aa6219aaa5267d1b4 in this term.
I20260620 18:31:51.529639 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cdd0fd6fcb842239d0fb74758cd1290" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4" is_pre_election: true
I20260620 18:31:51.530514 10978 leader_election.cc:302] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a4ce671d3984e1a8592bc29d45612bc; no voters: 7696506dcc9746d0aea9b1129a3e9433, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:31:51.531095 11169 raft_consensus.cc:3037] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
18:31:51.532 [HMS - ERROR - pool-6-thread-1] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_08795a50aec84f85bc35ae8025674185 already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy25.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
I20260620 18:31:51.534296 11169 raft_consensus.cc:2726] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:31:51.615020 11169 raft_consensus.cc:482] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:51.615460 11169 raft_consensus.cc:504] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.617149 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "654f8832c4be4d9ea5d23b446696a63f" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:31:51.617507 10750 raft_consensus.cc:2454] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 0.
I20260620 18:31:51.618144 10977 leader_election.cc:302] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc; no voters:
I20260620 18:31:51.618489 11196 raft_consensus.cc:2781] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:31:51.618618 11196 raft_consensus.cc:482] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:31:51.618726 11196 raft_consensus.cc:3037] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:51.618770 11169 leader_election.cc:288] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:31:51.622809 11196 raft_consensus.cc:504] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.623462 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "654f8832c4be4d9ea5d23b446696a63f" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4" is_pre_election: true
I20260620 18:31:51.623805 10899 raft_consensus.cc:2454] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 0.
I20260620 18:31:51.625171 11196 leader_election.cc:288] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 election: Requested vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:31:51.625902 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "654f8832c4be4d9ea5d23b446696a63f" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433"
I20260620 18:31:51.626204 10750 raft_consensus.cc:3037] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:51.626264 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "654f8832c4be4d9ea5d23b446696a63f" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4"
I20260620 18:31:51.626690 10899 raft_consensus.cc:3037] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:51.629384 10750 raft_consensus.cc:2454] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 1.
I20260620 18:31:51.630096 10977 leader_election.cc:302] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc; no voters:
I20260620 18:31:51.630514 11196 raft_consensus.cc:2781] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:31:51.630685 11196 raft_consensus.cc:686] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [term 1 LEADER]: Becoming Leader. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Running, Role: LEADER
I20260620 18:31:51.631058 11196 consensus_queue.cc:237] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:51.640837 10899 raft_consensus.cc:2454] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 1.
W20260620 18:31:51.655221 11168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:31:51.655704 11168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:31:51.655851 11168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:31:51.667954 11168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:31:51.668133 11168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:31:51.668205 11168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:31:51.668323 11168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:31:51.683320 11168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.231.212:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.253:44633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.231.253
--webserver_port=37199
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 17:42:20 UTC on ccycloud.quasar-rymxqh.root.comops.site
build id 4457681
ASAN enabled
I20260620 18:31:51.685443 11168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:31:51.688009 11168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:31:51.707266 11227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:51.707413 11228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:51.713766 11230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:31:51.715277 11168 server_base.cc:1034] running on GCE node
I20260620 18:31:51.716368 11168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:31:51.718104 11168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:31:51.719551 11168 hybrid_clock.cc:648] HybridClock initialized: now 1781980311719470 us; error 46 us; skew 500 ppm
I20260620 18:31:51.720036 11168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 18:31:51.728410 11168 webserver.cc:460] Webserver started at http://127.9.231.253:37199/ using document root <none> and password file <none>
I20260620 18:31:51.729919 11168 fs_manager.cc:359] Metadata directory not provided
I20260620 18:31:51.730136 11168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260620 18:31:51.737507 11098 connection.cc:535] client connection to 127.9.231.254:43203 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 18:31:51.738018 11168 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
W20260620 18:31:51.738452 11098 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.231.254:43203)
W20260620 18:31:51.739035 11098 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.9.231.254:43203)
W20260620 18:31:51.739714 11098 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.9.231.254:43203)
W20260620 18:31:51.740078 11098 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.9.231.254:43203)
W20260620 18:31:51.743317 11165 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:31:51.741141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.231.254:43203 (local address 127.9.231.194:39099)
0620 18:31:51.741744 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0620 18:31:51.741768 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 18:31:51.741921 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:31:51.742109 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":448}
W20260620 18:31:51.745820 11194 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:31:51.738417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.231.254:43203 (local address 127.9.231.195:55405)
0620 18:31:51.739939 (+ 1522us) negotiation.cc:107] Waiting for socket to connect
0620 18:31:51.739967 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0620 18:31:51.740147 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:31:51.744894 (+ 4747us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1269}
I20260620 18:31:51.747190 11237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:31:51.749456 11168 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.004s
I20260620 18:31:51.749647 11168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data,/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
uuid: "650e0197335644718cd91d5018399ee2"
format_stamp: "Formatted at 2026-06-20 18:31:46 on dist-test-slave-dv4w"
I20260620 18:31:51.750705 11168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:31:51.765141 11243 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1cd216a992124f0fac2fb65754e6f7df)
I20260620 18:31:51.765645 11243 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:51.767186 11243 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
W20260620 18:31:51.768836 10543 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:31:51.773767 10543 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:31:51.778275 10542 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
I20260620 18:31:51.778473 10542 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4587918520d849eaad95e35037e36608; no voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2
I20260620 18:31:51.778812 11243 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260620 18:31:51.790773 11168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:31:51.792116 11168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:31:51.792610 11168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:31:51.825057 11168 rpc_server.cc:307] RPC server started. Bound to: 127.9.231.253:44633
I20260620 18:31:51.825121 11293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.231.253:44633 every 8 connection(s)
I20260620 18:31:51.827270 11168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data/info.pb
I20260620 18:31:51.837301 10143 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu as pid 11168
I20260620 18:31:51.837962 11294 sys_catalog.cc:263] Verifying existing consensus state
I20260620 18:31:51.845705 11294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Bootstrap starting.
I20260620 18:31:51.864120 11294 log.cc:824] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Log is configured to *not* fsync() on all Append() calls
I20260620 18:31:51.869025 11298 consensus_queue.cc:1035] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:31:51.889959 10798 heartbeater.cc:347] Connected to a master server at 127.9.231.253:44633
I20260620 18:31:51.891741 11298 consensus_queue.cc:1035] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:31:51.900079 10946 heartbeater.cc:347] Connected to a master server at 127.9.231.253:44633
I20260620 18:31:51.912927 11301 consensus_queue.cc:1035] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:31:51.918607 11304 consensus_queue.cc:1035] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:31:51.933064 11294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:31:51.933845 11294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Bootstrap complete.
I20260620 18:31:51.937273 10946 heartbeater.cc:347] Connected to a master server at 127.9.231.253:44633
I20260620 18:31:51.938055 11089 heartbeater.cc:347] Connected to a master server at 127.9.231.253:44633
I20260620 18:31:51.938324 11089 heartbeater.cc:503] Master 127.9.231.253:44633 was elected leader, sending a full tablet report...
I20260620 18:31:51.942523 11089 heartbeater.cc:503] Master 127.9.231.253:44633 was elected leader, sending a full tablet report...
I20260620 18:31:51.946625 11294 raft_consensus.cc:348] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:51.947678 11294 raft_consensus.cc:729] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 650e0197335644718cd91d5018399ee2, State: Initialized, Role: FOLLOWER
I20260620 18:31:51.948416 11294 consensus_queue.cc:260] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:51.949442 11316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:51.949941 11316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:51.950603 11294 sys_catalog.cc:564] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:31:51.971728 11330 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:31:51.971875 11330 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: loading cluster ID for follower catalog manager: success
I20260620 18:31:51.964159 11168 master_runner.cc:186] Error getting master registration for 127.9.231.254:43203: Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111),
I20260620 18:31:51.976773 11330 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: acquiring CA information for follower catalog manager: success
I20260620 18:31:51.977228 11168 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:31:51.980633 11330 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:31:52.165004 11196 consensus_queue.cc:1035] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:31:52.169535 11167 consensus_queue.cc:1035] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:31:52.170589 11196 consensus_queue.cc:1035] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:31:52.182952 10522 master_runner.cc:186] Error getting master registration for 127.9.231.254:43203: Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111),
I20260620 18:31:52.183415 10522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:31:52.200981 11259 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" instance_seqno: 1781980308024586) as {username='slave'} at 127.9.231.193:50781; Asking this server to re-register.
I20260620 18:31:52.203711 10798 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:52.205284 11308 consensus_queue.cc:1035] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:31:52.206174 11259 ts_manager.cc:194] Registered new tserver with Master: 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961)
I20260620 18:31:52.207113 11258 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" instance_seqno: 1781980309047005) as {username='slave'} at 127.9.231.194:41039; Asking this server to re-register.
I20260620 18:31:52.208192 11089 heartbeater.cc:347] Connected to a master server at 127.9.231.253:44633
I20260620 18:31:52.208451 11089 heartbeater.cc:503] Master 127.9.231.253:44633 was elected leader, sending a full tablet report...
I20260620 18:31:52.210613 10946 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:52.210731 11258 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" instance_seqno: 1781980309668499) as {username='slave'} at 127.9.231.195:45291; Asking this server to re-register.
I20260620 18:31:52.212299 11089 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:52.213891 11167 consensus_queue.cc:1035] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:31:52.215067 11258 ts_manager.cc:194] Registered new tserver with Master: a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:31:52.216321 11259 ts_manager.cc:194] Registered new tserver with Master: 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:52.234632 11304 consensus_queue.cc:1035] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:31:52.297991 11343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1cd216a992124f0fac2fb65754e6f7df)
I20260620 18:31:52.298187 11343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:52.299638 11343 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
W20260620 18:31:52.302652 10542 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
I20260620 18:31:52.308380 11269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 6 candidate_status { last_received { term: 5 index: 26 } } ignore_live_leader: false dest_uuid: "650e0197335644718cd91d5018399ee2" is_pre_election: true
I20260620 18:31:52.309283 11269 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4587918520d849eaad95e35037e36608 in term 4.
I20260620 18:31:52.310148 10543 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2; no voters: 1cd216a992124f0fac2fb65754e6f7df
I20260620 18:31:52.310672 11343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260620 18:31:52.310839 11343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1cd216a992124f0fac2fb65754e6f7df)
I20260620 18:31:52.310966 11343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 5 FOLLOWER]: Advancing to term 6
I20260620 18:31:52.313452 11343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:52.314241 11343 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 6 election: Requested vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
I20260620 18:31:52.314986 11269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 6 candidate_status { last_received { term: 5 index: 26 } } ignore_live_leader: false dest_uuid: "650e0197335644718cd91d5018399ee2"
I20260620 18:31:52.315433 11269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 4 FOLLOWER]: Advancing to term 6
W20260620 18:31:52.318883 10542 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
I20260620 18:31:52.321645 11269 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4587918520d849eaad95e35037e36608 in term 6.
I20260620 18:31:52.322263 10543 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2; no voters: 1cd216a992124f0fac2fb65754e6f7df
I20260620 18:31:52.322636 11343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 6 FOLLOWER]: Leader election won for term 6
I20260620 18:31:52.323115 11343 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 6 LEADER]: Becoming Leader. State: Replica: 4587918520d849eaad95e35037e36608, State: Running, Role: LEADER
I20260620 18:31:52.323562 11343 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 5.26, Last appended by leader: 26, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:52.325145 11346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4587918520d849eaad95e35037e36608. Latest consensus state: current_term: 6 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:52.325466 11346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: LEADER
I20260620 18:31:52.326292 11348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:31:52.328918 11348 catalog_manager.cc:669] Loaded metadata for table default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]
I20260620 18:31:52.329475 11348 catalog_manager.cc:669] Loaded metadata for table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]
I20260620 18:31:52.329883 11348 catalog_manager.cc:669] Loaded metadata for table default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74]
I20260620 18:31:52.330291 11348 catalog_manager.cc:669] Loaded metadata for table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]
I20260620 18:31:52.330657 11348 catalog_manager.cc:669] Loaded metadata for table default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3]
I20260620 18:31:52.331038 11348 catalog_manager.cc:669] Loaded metadata for table default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59]
I20260620 18:31:52.331388 11348 catalog_manager.cc:669] Loaded metadata for table default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2]
I20260620 18:31:52.333350 11348 tablet_loader.cc:96] loaded metadata for tablet 1709fe2d11a24f52bd1812bc37e1714b (table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c])
I20260620 18:31:52.333917 11348 tablet_loader.cc:96] loaded metadata for tablet 4cdd0fd6fcb842239d0fb74758cd1290 (table default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2])
I20260620 18:31:52.334283 11348 tablet_loader.cc:96] loaded metadata for tablet 4fb0f7d77bac4b11bc0aa4486b39aa97 (table default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047])
I20260620 18:31:52.334641 11348 tablet_loader.cc:96] loaded metadata for tablet 5151602cea1443a9a56935c28db34beb (table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770])
I20260620 18:31:52.334987 11348 tablet_loader.cc:96] loaded metadata for tablet 654f8832c4be4d9ea5d23b446696a63f (table default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59])
I20260620 18:31:52.335302 11348 tablet_loader.cc:96] loaded metadata for tablet 8009d3fea30b4efa951e07d0bc2da1a5 (table default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74])
I20260620 18:31:52.335621 11348 tablet_loader.cc:96] loaded metadata for tablet b7a80fba1822404a98459c47faef4680 (table default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3])
I20260620 18:31:52.336076 11348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:31:52.337190 11348 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:31:52.337302 11348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:31:52.338635 11348 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:31:52.339597 11348 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Loaded TSK: 0
I20260620 18:31:52.340168 11348 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:31:52.341657 11348 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260620 18:31:52.342905 11348 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260620 18:31:52.360116 10557 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53880:
name: "default.Table_fce67ec520194c7fabdca1fce51af8a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:52.360620 10554 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53880:
name: "default.Table_e1ad292ef5e24a628dcf2d699e127e57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:52.361562 10552 master_service.cc:946] ReplaceTablet: received request to replace tablet 8009d3fea30b4efa951e07d0bc2da1a5 from {username='slave'} at 127.0.0.1:53880
I20260620 18:31:52.361500 10555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53880:
name: "default.Table_e4c5c6935f7a48cebd0f828771b1fe01"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:52.362363 10556 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53880:
table { table_name: "default.Table_88afa4cb37564293a220876026ee1931" } modify_external_catalogs: true
I20260620 18:31:52.362555 10550 master_service.cc:946] ReplaceTablet: received request to replace tablet b7a80fba1822404a98459c47faef4680 from {username='slave'} at 127.0.0.1:53880
I20260620 18:31:52.362636 10556 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53880:
table { table_name: "default.Table_88afa4cb37564293a220876026ee1931" } modify_external_catalogs: true
I20260620 18:31:52.362628 10549 master_service.cc:946] ReplaceTablet: received request to replace tablet b7a80fba1822404a98459c47faef4680 from {username='slave'} at 127.0.0.1:53880
I20260620 18:31:52.363157 10548 master_service.cc:946] ReplaceTablet: received request to replace tablet 4cdd0fd6fcb842239d0fb74758cd1290 from {username='slave'} at 127.0.0.1:53880
I20260620 18:31:52.363060 10551 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:53880:
table { table_name: "default.Table_9e9a3aec7a67407ba81fe1f6797b55f6" } new_table_name: "default.Table_70f8c2fd535343bfb5649c75f207510a" modify_external_catalogs: true
I20260620 18:31:52.363600 10553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53880:
table { table_name: "default.Table_f27c96a5d11944f6a75123e42b5b2ce5" } modify_external_catalogs: true
I20260620 18:31:52.363780 10553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53880:
table { table_name: "default.Table_f27c96a5d11944f6a75123e42b5b2ce5" } modify_external_catalogs: true
I20260620 18:31:52.373489 11269 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 6 FOLLOWER]: Refusing update from remote peer 4587918520d849eaad95e35037e36608: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 28. (index mismatch)
W20260620 18:31:52.374084 10542 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 -> Peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Couldn't send request to peer 1cd216a992124f0fac2fb65754e6f7df. Status: Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 18:31:52.374421 11343 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [LEADER]: Connected to new peer: Peer: permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 5, Time since last communication: 0.000s
F20260620 18:31:52.381353 10631 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_269d7d1648864d4e8b29b944b7115e6a
*** Check failure stack trace: ***
@ 0x7f27390dd9ad google::LogMessage::Fail() at ??:0
@ 0x7f27390e088a google::LogMessage::SendToLog() at ??:0
@ 0x7f27390dd4de google::LogMessage::Flush() at ??:0
@ 0x7f27390dd6d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f274ec7bba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f274ec7a901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f274ec8166c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260620 18:31:52.415421 11354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:52.415910 11354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:52.416333 11316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4587918520d849eaad95e35037e36608. Latest consensus state: current_term: 6 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:52.416646 11316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:52.419510 11316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:52.419857 11316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:52.420100 11339 consensus_queue.cc:1035] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:31:52.435144 11338 consensus_queue.cc:1035] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
@ 0x7f274ec814e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260620 18:31:52.454937 11346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:52.455396 11346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: LEADER
I20260620 18:31:52.458518 10552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Sending DeleteTablet for 3 replicas of tablet 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:31:52.459254 10548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Sending DeleteTablet for 3 replicas of tablet 4cdd0fd6fcb842239d0fb74758cd1290
I20260620 18:31:52.461392 10548 catalog_manager.cc:6565] ReplaceTablet: tablet 4cdd0fd6fcb842239d0fb74758cd1290 deleted and replaced by tablet 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:31:52.467631 10552 catalog_manager.cc:6565] ReplaceTablet: tablet 8009d3fea30b4efa951e07d0bc2da1a5 deleted and replaced by tablet acc2a8b335d14d20bad1719566a27f3d
@ 0x7f273bd07eaf kudu::Thread::SuperviseThread() at ??:0
I20260620 18:31:52.483116 10550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Sending DeleteTablet for 3 replicas of tablet b7a80fba1822404a98459c47faef4680
I20260620 18:31:52.483767 10550 catalog_manager.cc:6565] ReplaceTablet: tablet b7a80fba1822404a98459c47faef4680 deleted and replaced by tablet 6a00bd448f6e4d1fbf99e4c11bb6e970
I20260620 18:31:52.485289 10549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53880:
name: "default.Table_5854ef7ddc554da59e75ed6ff18718ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:52.498651 10731 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cdd0fd6fcb842239d0fb74758cd1290 with delete_type TABLET_DATA_DELETED (replaced by tablet 8b82c74e9aa6451c8e7c537f7096aa25) from {username='slave'} at 127.0.0.1:57720
I20260620 18:31:52.499045 10729 tablet_service.cc:1514] Processing DeleteTablet for tablet b7a80fba1822404a98459c47faef4680 with delete_type TABLET_DATA_DELETED (replaced by tablet 6a00bd448f6e4d1fbf99e4c11bb6e970) from {username='slave'} at 127.0.0.1:57720
I20260620 18:31:52.499274 10730 tablet_service.cc:1514] Processing DeleteTablet for tablet 8009d3fea30b4efa951e07d0bc2da1a5 with delete_type TABLET_DATA_DELETED (replaced by tablet acc2a8b335d14d20bad1719566a27f3d) from {username='slave'} at 127.0.0.1:57720
I20260620 18:31:52.499825 10727 tablet_service.cc:1467] Processing CreateTablet for tablet acc2a8b335d14d20bad1719566a27f3d (DEFAULT_TABLE table=default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:52.499985 10728 tablet_service.cc:1467] Processing CreateTablet for tablet 8b82c74e9aa6451c8e7c537f7096aa25 (DEFAULT_TABLE table=default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:52.500461 10727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acc2a8b335d14d20bad1719566a27f3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:52.500768 10728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b82c74e9aa6451c8e7c537f7096aa25. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f273efe16db start_thread at ??:0
@ 0x7f2737f7761f clone at ??:0
I20260620 18:31:52.505451 11374 tablet_replica.cc:330] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433: stopping tablet replica
I20260620 18:31:52.505733 11374 raft_consensus.cc:2229] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:31:52.515601 11374 raft_consensus.cc:2258] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:31:52.521064 11374 ts_tablet_manager.cc:1905] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:31:52.527706 11350 mvcc.cc:205] Tried to move back new op lower bound from 7298991359470915584 to 7298991349375451136. Current Snapshot: MvccSnapshot[applied={T|T < 7298991359470915584}]
I20260620 18:31:52.536500 11375 tablet_bootstrap.cc:492] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:31:52.538842 11375 tablet_bootstrap.cc:654] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:52.542387 11374 ts_tablet_manager.cc:1918] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:31:52.542611 11374 log.cc:1196] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal/wals/4cdd0fd6fcb842239d0fb74758cd1290
I20260620 18:31:52.543110 11374 ts_tablet_manager.cc:1939] T 4cdd0fd6fcb842239d0fb74758cd1290 P 7696506dcc9746d0aea9b1129a3e9433: Deleting consensus metadata
I20260620 18:31:52.544607 11374 tablet_replica.cc:330] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433: stopping tablet replica
I20260620 18:31:52.544945 11374 raft_consensus.cc:2229] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:31:52.545498 11374 raft_consensus.cc:2258] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:31:52.546800 11374 ts_tablet_manager.cc:1905] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:31:52.547909 11375 tablet_bootstrap.cc:492] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:31:52.548142 11375 ts_tablet_manager.cc:1397] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260620 18:31:52.549134 11375 raft_consensus.cc:348] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:52.549554 11375 raft_consensus.cc:374] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:52.549716 11375 raft_consensus.cc:729] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:31:52.550069 11375 consensus_queue.cc:260] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:52.552845 11375 ts_tablet_manager.cc:1428] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260620 18:31:52.553265 11375 tablet_bootstrap.cc:492] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:31:52.556164 11375 tablet_bootstrap.cc:654] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:52.557020 11374 ts_tablet_manager.cc:1918] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:31:52.557266 11374 log.cc:1196] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal/wals/b7a80fba1822404a98459c47faef4680
I20260620 18:31:52.557801 11374 ts_tablet_manager.cc:1939] T b7a80fba1822404a98459c47faef4680 P 7696506dcc9746d0aea9b1129a3e9433: Deleting consensus metadata
I20260620 18:31:52.558746 11375 tablet_bootstrap.cc:492] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:31:52.558848 11374 tablet_replica.cc:330] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433: stopping tablet replica
I20260620 18:31:52.558982 11375 ts_tablet_manager.cc:1397] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 18:31:52.559114 11374 raft_consensus.cc:2229] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:31:52.559333 11374 raft_consensus.cc:2258] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:31:52.559855 11375 raft_consensus.cc:348] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:52.560104 11374 ts_tablet_manager.cc:1905] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:31:52.560105 11375 raft_consensus.cc:374] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:52.560240 11375 raft_consensus.cc:729] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:31:52.560571 11375 consensus_queue.cc:260] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:52.562886 11375 ts_tablet_manager.cc:1428] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260620 18:31:52.567781 11374 ts_tablet_manager.cc:1918] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:31:52.567906 11374 log.cc:1196] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal/wals/8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:31:52.568220 11374 ts_tablet_manager.cc:1939] T 8009d3fea30b4efa951e07d0bc2da1a5 P 7696506dcc9746d0aea9b1129a3e9433: Deleting consensus metadata
I20260620 18:31:52.616406 11339 consensus_queue.cc:1035] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:31:52.622799 11196 consensus_queue.cc:1035] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 18:31:52.728549 11099 connection.cc:535] client connection to 127.9.231.252:34267 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:31:52.729154 11371 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:31:52.464047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37234 (local address 127.9.231.194:45409)
0620 18:31:52.505288 (+ 41241us) server_negotiation.cc:206] Beginning negotiation
0620 18:31:52.505295 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 18:31:52.505320 (+ 25us) server_negotiation.cc:407] Connection header received
0620 18:31:52.505398 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:31:52.505403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:31:52.505483 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:31:52.505581 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:31:52.728714 (+223133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37234: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":41092,"thread_start_us":42089,"threads_started":1}
W20260620 18:31:52.729221 11369 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:31:52.462700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49162 (local address 127.9.231.195:42583)
0620 18:31:52.516356 (+ 53656us) server_negotiation.cc:206] Beginning negotiation
0620 18:31:52.516364 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 18:31:52.516402 (+ 38us) server_negotiation.cc:407] Connection header received
0620 18:31:52.516510 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:31:52.516515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:31:52.516616 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:31:52.516779 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:31:52.728790 (+212011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49162: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":53464,"thread_start_us":61591,"threads_started":1}
W20260620 18:31:52.729955 10684 connection.cc:535] server connection from 127.0.0.1:57720 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:31:52.730805 11239 connection.cc:535] server connection from 127.0.0.1:38666 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 18:31:52.836869 11167 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:52.837164 11167 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:52.837859 11167 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:31:52.839217 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:31:52.839445 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:31:52.839596 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:31:52.839937 11167 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:31:52.915064 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 1: this message will repeat every 5th retry.
I20260620 18:31:53.002367 11388 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:31:53.002682 11388 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:53.005637 11388 leader_election.cc:288] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:31:53.007109 11240 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
W20260620 18:31:53.009071 11240 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
W20260620 18:31:53.009419 11241 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:31:53.009567 11241 leader_election.cc:302] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 650e0197335644718cd91d5018399ee2; no voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608
I20260620 18:31:53.010130 11388 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260620 18:31:53.068563 11167 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:53.068848 11167 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:53.069638 11167 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:31:53.071044 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:31:53.071359 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:31:53.071465 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:31:53.071802 11167 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:31:53.263541 10977 consensus_peers.cc:487] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5. This is attempt 1: this message will repeat every 5th retry.
I20260620 18:31:53.628242 11396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:31:53.628508 11396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:53.629272 11396 leader_election.cc:288] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:31:53.632893 11240 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
W20260620 18:31:53.633186 11241 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:31:53.633486 11241 leader_election.cc:302] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 650e0197335644718cd91d5018399ee2; no voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608
I20260620 18:31:53.633836 11396 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260620 18:31:54.219328 11407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:31:54.219586 11407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:54.220382 11407 leader_election.cc:288] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:31:54.222975 11241 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:31:54.223218 11240 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
I20260620 18:31:54.223380 11240 leader_election.cc:302] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 650e0197335644718cd91d5018399ee2; no voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608
I20260620 18:31:54.223765 11407 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260620 18:31:54.686338 10143 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
/tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.252:34267
--webserver_interface=127.9.231.252
--webserver_port=42823
--builtin_ntp_servers=127.9.231.212:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260620 18:31:55.099643 11416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:31:55.100163 11416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:31:55.100363 11416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:31:55.109131 11416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:31:55.109308 11416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:31:55.109413 11416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:31:55.109519 11416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:31:55.120761 11416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.231.212:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.252:34267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.231.252
--webserver_port=42823
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 17:42:20 UTC on ccycloud.quasar-rymxqh.root.comops.site
build id 4457681
ASAN enabled
I20260620 18:31:55.122601 11416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:31:55.125069 11416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:31:55.142026 11425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:55.142069 11426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:55.143405 11428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:31:55.146656 11416 server_base.cc:1034] running on GCE node
I20260620 18:31:55.148159 11416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:31:55.151042 11416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:31:55.152477 11416 hybrid_clock.cc:648] HybridClock initialized: now 1781980315152388 us; error 83 us; skew 500 ppm
I20260620 18:31:55.153080 11416 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 18:31:55.157112 11416 webserver.cc:460] Webserver started at http://127.9.231.252:42823/ using document root <none> and password file <none>
I20260620 18:31:55.158375 11416 fs_manager.cc:359] Metadata directory not provided
I20260620 18:31:55.158573 11416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:31:55.166384 11416 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 18:31:55.171099 11434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:31:55.173674 11416 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260620 18:31:55.173990 11416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data,/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/wal
uuid: "4587918520d849eaad95e35037e36608"
format_stamp: "Formatted at 2026-06-20 18:31:46 on dist-test-slave-dv4w"
I20260620 18:31:55.175347 11416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:31:55.202754 11416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:31:55.203956 11416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:31:55.204416 11416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:31:55.216470 11439 raft_consensus.cc:482] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7696506dcc9746d0aea9b1129a3e9433)
I20260620 18:31:55.216714 11439 raft_consensus.cc:504] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:55.217445 11439 leader_election.cc:288] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:31:55.218189 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a80fba1822404a98459c47faef4680" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4" is_pre_election: true
W20260620 18:31:55.218647 10977 leader_election.cc:341] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: b7a80fba1822404a98459c47faef4680
I20260620 18:31:55.219100 10978 leader_election.cc:302] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a4ce671d3984e1a8592bc29d45612bc; no voters: 7696506dcc9746d0aea9b1129a3e9433, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:31:55.219370 11439 raft_consensus.cc:2726] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:31:55.220340 11420 raft_consensus.cc:482] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7696506dcc9746d0aea9b1129a3e9433)
I20260620 18:31:55.220515 11420 raft_consensus.cc:504] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:55.221293 11420 leader_election.cc:288] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:55.222066 11041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a80fba1822404a98459c47faef4680" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc" is_pre_election: true
W20260620 18:31:55.222199 10832 leader_election.cc:341] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: b7a80fba1822404a98459c47faef4680
I20260620 18:31:55.222357 11041 raft_consensus.cc:2454] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
I20260620 18:31:55.222776 10833 leader_election.cc:302] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433
I20260620 18:31:55.223137 11420 raft_consensus.cc:2781] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 18:31:55.223497 11420 raft_consensus.cc:482] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7696506dcc9746d0aea9b1129a3e9433)
I20260620 18:31:55.223649 11420 raft_consensus.cc:3037] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Advancing to term 2
I20260620 18:31:55.227090 11420 raft_consensus.cc:504] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:55.227908 11420 leader_election.cc:288] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 election: Requested vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:55.228607 11041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a80fba1822404a98459c47faef4680" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc"
I20260620 18:31:55.228864 11041 raft_consensus.cc:3037] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Advancing to term 2
W20260620 18:31:55.229242 10832 leader_election.cc:341] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: b7a80fba1822404a98459c47faef4680
I20260620 18:31:55.229985 11416 rpc_server.cc:307] RPC server started. Bound to: 127.9.231.252:34267
I20260620 18:31:55.230005 11486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.231.252:34267 every 8 connection(s)
I20260620 18:31:55.231294 11041 raft_consensus.cc:2454] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 2.
I20260620 18:31:55.231869 10833 leader_election.cc:302] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433
I20260620 18:31:55.232095 11416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data/info.pb
I20260620 18:31:55.232411 11420 raft_consensus.cc:2781] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 18:31:55.232677 11420 raft_consensus.cc:686] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [term 2 LEADER]: Becoming Leader. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Running, Role: LEADER
I20260620 18:31:55.232837 10143 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu as pid 11416
I20260620 18:31:55.233392 11420 consensus_queue.cc:237] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:55.254688 11487 sys_catalog.cc:263] Verifying existing consensus state
I20260620 18:31:55.261449 11487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Bootstrap starting.
I20260620 18:31:55.266973 10944 heartbeater.cc:347] Connected to a master server at 127.9.231.252:34267
I20260620 18:31:55.266963 10796 heartbeater.cc:347] Connected to a master server at 127.9.231.252:34267
I20260620 18:31:55.278352 11487 log.cc:824] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Log is configured to *not* fsync() on all Append() calls
I20260620 18:31:55.310141 11487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Bootstrap replayed 1/1 log segments. Stats: ops{read=33 overwritten=1 applied=31 ignored=0} inserts{seen=22 ignored=0} mutations{seen=19 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260620 18:31:55.310969 11487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Bootstrap complete.
I20260620 18:31:55.321004 11487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 6 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:55.325604 11487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4587918520d849eaad95e35037e36608, State: Initialized, Role: FOLLOWER
I20260620 18:31:55.326783 11487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 6.32, Last appended by leader: 32, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:55.328138 11495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:55.328644 11495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:55.329668 11487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:31:55.348235 11509 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:31:55.348498 11509 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: loading cluster ID for follower catalog manager: success
I20260620 18:31:55.353780 11509 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: acquiring CA information for follower catalog manager: success
I20260620 18:31:55.358237 11509 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:31:55.351999 11416 master_runner.cc:186] Error getting master registration for 127.9.231.254:43203: Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111),
I20260620 18:31:55.362476 11416 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:31:55.440371 11087 heartbeater.cc:347] Connected to a master server at 127.9.231.252:34267
I20260620 18:31:55.444307 11451 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" instance_seqno: 1781980309668499) as {username='slave'} at 127.9.231.195:42285; Asking this server to re-register.
I20260620 18:31:55.445310 11087 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:55.447659 11451 ts_manager.cc:194] Registered new tserver with Master: 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:55.712077 11515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:31:55.712455 11515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:55.713393 11515 leader_election.cc:288] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:31:55.716555 11240 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
I20260620 18:31:55.723385 11462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "650e0197335644718cd91d5018399ee2" candidate_term: 7 candidate_status { last_received { term: 6 index: 32 } } ignore_live_leader: false dest_uuid: "4587918520d849eaad95e35037e36608" is_pre_election: true
I20260620 18:31:55.724159 11462 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 650e0197335644718cd91d5018399ee2 in term 6.
I20260620 18:31:55.725119 11241 leader_election.cc:302] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2; no voters: 1cd216a992124f0fac2fb65754e6f7df
I20260620 18:31:55.725728 11515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260620 18:31:55.725894 11515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:31:55.726006 11515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 6 FOLLOWER]: Advancing to term 7
I20260620 18:31:55.728377 11515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:55.729384 11515 leader_election.cc:288] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 election: Requested vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
I20260620 18:31:55.730378 11462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "650e0197335644718cd91d5018399ee2" candidate_term: 7 candidate_status { last_received { term: 6 index: 32 } } ignore_live_leader: false dest_uuid: "4587918520d849eaad95e35037e36608"
I20260620 18:31:55.730697 11462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 6 FOLLOWER]: Advancing to term 7
W20260620 18:31:55.731895 11240 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
I20260620 18:31:55.737995 11462 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 650e0197335644718cd91d5018399ee2 in term 7.
I20260620 18:31:55.738792 11241 leader_election.cc:302] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2; no voters: 1cd216a992124f0fac2fb65754e6f7df
I20260620 18:31:55.739300 11515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 7 FOLLOWER]: Leader election won for term 7
I20260620 18:31:55.740026 11515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 7 LEADER]: Becoming Leader. State: Replica: 650e0197335644718cd91d5018399ee2, State: Running, Role: LEADER
I20260620 18:31:55.740633 11515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 6.32, Last appended by leader: 32, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:55.742807 11519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 650e0197335644718cd91d5018399ee2. Latest consensus state: current_term: 7 leader_uuid: "650e0197335644718cd91d5018399ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:55.743333 11519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: LEADER
I20260620 18:31:55.772795 11451 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" instance_seqno: 1781980308024586) as {username='slave'} at 127.9.231.193:37355; Asking this server to re-register.
I20260620 18:31:55.772981 11450 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" instance_seqno: 1781980309047005) as {username='slave'} at 127.9.231.194:39503; Asking this server to re-register.
I20260620 18:31:55.773696 10796 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:55.773936 10944 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:55.775491 11450 ts_manager.cc:194] Registered new tserver with Master: 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961)
I20260620 18:31:55.775563 11451 ts_manager.cc:194] Registered new tserver with Master: a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:31:56.133903 11041 raft_consensus.cc:1261] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 2 FOLLOWER]: Refusing update from remote peer a0ae894b0758446aa6219aaa5267d1b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260620 18:31:56.134845 11523 consensus_queue.cc:1035] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260620 18:31:56.144533 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 1: this message will repeat every 5th retry.
W20260620 18:31:56.284788 11240 consensus_peers.cc:487] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 -> Peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Couldn't send request to peer 1cd216a992124f0fac2fb65754e6f7df. Status: Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 18:31:56.308859 11532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:56.309129 11532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:56.312114 11532 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
W20260620 18:31:56.314061 11436 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
I20260620 18:31:56.314514 11536 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:56.315047 11536 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:56.315990 11536 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:31:56.316424 11436 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
W20260620 18:31:56.317143 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:31:56.317400 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:31:56.317548 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:31:56.317847 11536 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:31:56.321815 11269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 8 candidate_status { last_received { term: 6 index: 32 } } ignore_live_leader: false dest_uuid: "650e0197335644718cd91d5018399ee2" is_pre_election: true
I20260620 18:31:56.323287 11437 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4587918520d849eaad95e35037e36608; no voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2
I20260620 18:31:56.323822 11532 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260620 18:31:56.350250 11462 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Refusing update from remote peer 650e0197335644718cd91d5018399ee2: Log matching property violated. Preceding OpId in replica: term: 6 index: 32. Preceding OpId from leader: term: 7 index: 33. (index mismatch)
I20260620 18:31:56.351495 11529 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 31, Time since last communication: 0.000s
I20260620 18:31:56.355093 11532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 650e0197335644718cd91d5018399ee2. Latest consensus state: current_term: 7 leader_uuid: "650e0197335644718cd91d5018399ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:56.355433 11532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:56.361021 11532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "650e0197335644718cd91d5018399ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:56.361253 11532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:56.361887 11529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "650e0197335644718cd91d5018399ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:56.362208 11529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: LEADER
I20260620 18:31:56.390676 11521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:31:56.393632 11521 catalog_manager.cc:669] Loaded metadata for table default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]
I20260620 18:31:56.394119 11521 catalog_manager.cc:669] Loaded metadata for table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]
I20260620 18:31:56.394488 11521 catalog_manager.cc:669] Loaded metadata for table default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74]
I20260620 18:31:56.394846 11521 catalog_manager.cc:669] Loaded metadata for table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]
I20260620 18:31:56.395212 11521 catalog_manager.cc:669] Loaded metadata for table default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3]
I20260620 18:31:56.395583 11521 catalog_manager.cc:669] Loaded metadata for table default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59]
I20260620 18:31:56.395920 11521 catalog_manager.cc:669] Loaded metadata for table default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2]
I20260620 18:31:56.397862 11521 tablet_loader.cc:96] loaded metadata for tablet 1709fe2d11a24f52bd1812bc37e1714b (table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c])
I20260620 18:31:56.398727 11521 tablet_loader.cc:96] loaded metadata for tablet 4fb0f7d77bac4b11bc0aa4486b39aa97 (table default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047])
I20260620 18:31:56.399066 11521 tablet_loader.cc:96] loaded metadata for tablet 5151602cea1443a9a56935c28db34beb (table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770])
I20260620 18:31:56.399377 11521 tablet_loader.cc:96] loaded metadata for tablet 654f8832c4be4d9ea5d23b446696a63f (table default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59])
I20260620 18:31:56.399675 11521 tablet_loader.cc:96] loaded metadata for tablet 6a00bd448f6e4d1fbf99e4c11bb6e970 (table default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3])
I20260620 18:31:56.400092 11521 tablet_loader.cc:96] loaded metadata for tablet 8b82c74e9aa6451c8e7c537f7096aa25 (table default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2])
I20260620 18:31:56.400478 11521 tablet_loader.cc:96] loaded metadata for tablet acc2a8b335d14d20bad1719566a27f3d (table default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74])
I20260620 18:31:56.401063 11521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:31:56.402140 11521 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:31:56.402232 11521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:31:56.403571 11521 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:31:56.404618 11521 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Loaded TSK: 0
I20260620 18:31:56.405561 11521 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:31:56.406486 11521 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260620 18:31:56.407351 11521 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260620 18:31:56.479485 11258 master_service.cc:946] ReplaceTablet: received request to replace tablet 8009d3fea30b4efa951e07d0bc2da1a5 from {username='slave'} at 127.0.0.1:38646
I20260620 18:31:56.479816 11259 master_service.cc:946] ReplaceTablet: received request to replace tablet 5151602cea1443a9a56935c28db34beb from {username='slave'} at 127.0.0.1:38646
I20260620 18:31:56.480625 11257 master_service.cc:946] ReplaceTablet: received request to replace tablet 4cdd0fd6fcb842239d0fb74758cd1290 from {username='slave'} at 127.0.0.1:38646
I20260620 18:31:56.480878 11254 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38646:
table { table_name: "default.Table_be97363936284e2faa8400049925a98c" } new_table_name: "default.Table_4e545c1bf96b40de8e03778dfe0e4861" modify_external_catalogs: true
I20260620 18:31:56.481410 11256 master_service.cc:946] ReplaceTablet: received request to replace tablet b7a80fba1822404a98459c47faef4680 from {username='slave'} at 127.0.0.1:38646
I20260620 18:31:56.481220 11255 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38646:
name: "default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:56.482649 11256 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38646:
table { table_name: "default.Table_009e06deb1f84f74b4a32287a8c44b5a" } new_table_name: "default.Table_7105b83c587d4cf0a6add4272b0defc6" modify_external_catalogs: true
I20260620 18:31:56.483004 11252 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38646:
table { table_name: "default.Table_f27c96a5d11944f6a75123e42b5b2ce5" } modify_external_catalogs: true
I20260620 18:31:56.483399 11252 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38646:
table { table_name: "default.Table_f27c96a5d11944f6a75123e42b5b2ce5" } modify_external_catalogs: true
I20260620 18:31:56.483306 11257 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38646:
table { table_name: "default.Table_269d7d1648864d4e8b29b944b7115e6a" } new_table_name: "default.Table_45bc016e4a9b4288a42f33020e656b17" modify_external_catalogs: true
I20260620 18:31:56.484053 11258 master_service.cc:946] ReplaceTablet: received request to replace tablet b7a80fba1822404a98459c47faef4680 from {username='slave'} at 127.0.0.1:38646
I20260620 18:31:56.484447 11251 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38646:
name: "default.Table_e1ad292ef5e24a628dcf2d699e127e57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:56.485529 11250 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38646:
name: "default.Table_fce67ec520194c7fabdca1fce51af8a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:56.486336 11253 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38646:
name: "default.Table_e001ba7ef7b44f09b6262dc3d55d0df0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:31:56.486747 11258 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38646:
table { table_name: "default.Table_88afa4cb37564293a220876026ee1931" } modify_external_catalogs: true
I20260620 18:31:56.487013 11258 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38646:
table { table_name: "default.Table_88afa4cb37564293a220876026ee1931" } modify_external_catalogs: true
F20260620 18:31:56.491827 11329 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_269d7d1648864d4e8b29b944b7115e6a
*** Check failure stack trace: ***
I20260620 18:31:56.493610 11259 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Sending DeleteTablet for 3 replicas of tablet 5151602cea1443a9a56935c28db34beb
@ 0x7f52618ec9ad google::LogMessage::Fail() at ??:0
@ 0x7f52618ef88a google::LogMessage::SendToLog() at ??:0
I20260620 18:31:56.497383 11259 catalog_manager.cc:6565] ReplaceTablet: tablet 5151602cea1443a9a56935c28db34beb deleted and replaced by tablet fdce1a08f6e34686a1a50676ada66941
I20260620 18:31:56.499220 11259 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38646:
name: "default.Table_e4c5c6935f7a48cebd0f828771b1fe01"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f52618ec4de google::LogMessage::Flush() at ??:0
@ 0x7f52618ec6d1 google::LogMessage::~LogMessage() at ??:0
I20260620 18:31:56.512213 10728 tablet_service.cc:1514] Processing DeleteTablet for tablet 5151602cea1443a9a56935c28db34beb with delete_type TABLET_DATA_DELETED (replaced by tablet fdce1a08f6e34686a1a50676ada66941) from {username='slave'} at 127.0.0.1:57722
I20260620 18:31:56.514679 10727 tablet_service.cc:1467] Processing CreateTablet for tablet fdce1a08f6e34686a1a50676ada66941 (DEFAULT_TABLE table=default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:56.515313 10727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdce1a08f6e34686a1a50676ada66941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:56.523422 11536 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:56.523692 11536 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:31:56.524740 11536 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:31:56.526574 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:31:56.526777 11553 tablet_bootstrap.cc:492] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
W20260620 18:31:56.527204 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:31:56.527372 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:31:56.527963 11536 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:31:56.531077 11551 tablet_replica.cc:330] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433: stopping tablet replica
I20260620 18:31:56.531337 11551 raft_consensus.cc:2229] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:31:56.531581 11551 raft_consensus.cc:2258] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:31:56.532285 11551 ts_tablet_manager.cc:1905] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:31:56.534060 11553 tablet_bootstrap.cc:654] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
@ 0x7f527748aba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260620 18:31:56.538578 10878 tablet_service.cc:1514] Processing DeleteTablet for tablet 5151602cea1443a9a56935c28db34beb with delete_type TABLET_DATA_DELETED (replaced by tablet fdce1a08f6e34686a1a50676ada66941) from {username='slave'} at 127.0.0.1:37240
I20260620 18:31:56.540107 10879 tablet_service.cc:1467] Processing CreateTablet for tablet fdce1a08f6e34686a1a50676ada66941 (DEFAULT_TABLE table=default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:56.540953 10879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdce1a08f6e34686a1a50676ada66941. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f5277489901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260620 18:31:56.544045 11551 ts_tablet_manager.cc:1918] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:31:56.544384 11551 log.cc:1196] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal/wals/5151602cea1443a9a56935c28db34beb
I20260620 18:31:56.545017 11551 ts_tablet_manager.cc:1939] T 5151602cea1443a9a56935c28db34beb P 7696506dcc9746d0aea9b1129a3e9433: Deleting consensus metadata
I20260620 18:31:56.546976 11241 catalog_manager.cc:4841] TS 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): tablet 5151602cea1443a9a56935c28db34beb (table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]) successfully deleted
@ 0x7f527749066c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260620 18:31:56.551412 11557 tablet_replica.cc:330] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4: stopping tablet replica
I20260620 18:31:56.551872 11557 raft_consensus.cc:2229] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:31:56.552063 11558 tablet_bootstrap.cc:492] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
I20260620 18:31:56.552237 11557 raft_consensus.cc:2258] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:31:56.553350 11557 ts_tablet_manager.cc:1905] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f52774904e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260620 18:31:56.554590 11558 tablet_bootstrap.cc:654] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:56.555176 11553 tablet_bootstrap.cc:492] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:31:56.555343 11553 ts_tablet_manager.cc:1397] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.029s user 0.007s sys 0.000s
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260620 18:31:56.556325 11553 raft_consensus.cc:348] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:56.556696 11553 raft_consensus.cc:374] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:56.556795 11553 raft_consensus.cc:729] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:31:56.557076 11553 consensus_queue.cc:260] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:56.557811 11553 ts_tablet_manager.cc:1428] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
@ 0x7f5264516eaf kudu::Thread::SuperviseThread() at ??:0
I20260620 18:31:56.563795 11021 tablet_service.cc:1514] Processing DeleteTablet for tablet 5151602cea1443a9a56935c28db34beb with delete_type TABLET_DATA_DELETED (replaced by tablet fdce1a08f6e34686a1a50676ada66941) from {username='slave'} at 127.0.0.1:49166
I20260620 18:31:56.564150 11022 tablet_service.cc:1467] Processing CreateTablet for tablet fdce1a08f6e34686a1a50676ada66941 (DEFAULT_TABLE table=default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:31:56.564937 11022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdce1a08f6e34686a1a50676ada66941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:31:56.569169 11557 ts_tablet_manager.cc:1918] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:31:56.569326 11557 log.cc:1196] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/wal/wals/5151602cea1443a9a56935c28db34beb
I20260620 18:31:56.569403 11558 tablet_bootstrap.cc:492] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:31:56.569566 11558 ts_tablet_manager.cc:1397] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.000s
I20260620 18:31:56.569862 11557 ts_tablet_manager.cc:1939] T 5151602cea1443a9a56935c28db34beb P a0ae894b0758446aa6219aaa5267d1b4: Deleting consensus metadata
I20260620 18:31:56.570492 11558 raft_consensus.cc:348] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:56.571002 11558 raft_consensus.cc:374] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:56.571115 11558 raft_consensus.cc:729] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:31:56.571400 11242 catalog_manager.cc:4841] TS a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): tablet 5151602cea1443a9a56935c28db34beb (table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]) successfully deleted
I20260620 18:31:56.571425 11558 consensus_queue.cc:260] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
@ 0x7f52677f06db start_thread at ??:0
I20260620 18:31:56.575958 11558 ts_tablet_manager.cc:1428] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
@ 0x7f526078661f clone at ??:0
I20260620 18:31:56.580487 11561 tablet_replica.cc:330] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc: stopping tablet replica
I20260620 18:31:56.580758 11561 raft_consensus.cc:2229] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:31:56.581532 11561 raft_consensus.cc:2258] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:31:56.581884 11563 tablet_bootstrap.cc:492] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
I20260620 18:31:56.582291 11561 ts_tablet_manager.cc:1905] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:31:56.584527 11563 tablet_bootstrap.cc:654] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:31:56.588529 11563 tablet_bootstrap.cc:492] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:31:56.588711 11563 ts_tablet_manager.cc:1397] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20260620 18:31:56.589614 11563 raft_consensus.cc:348] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:56.589865 11563 raft_consensus.cc:374] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:31:56.589949 11563 raft_consensus.cc:729] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:31:56.590240 11563 consensus_queue.cc:260] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:56.590612 11561 ts_tablet_manager.cc:1918] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:31:56.590736 11561 log.cc:1196] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal/wals/5151602cea1443a9a56935c28db34beb
I20260620 18:31:56.591255 11561 ts_tablet_manager.cc:1939] T 5151602cea1443a9a56935c28db34beb P 9a4ce671d3984e1a8592bc29d45612bc: Deleting consensus metadata
I20260620 18:31:56.593822 11563 ts_tablet_manager.cc:1428] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
W20260620 18:31:56.809264 10975 connection.cc:535] server connection from 127.0.0.1:49166 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:31:56.809973 10977 connection.cc:535] client connection to 127.9.231.253:44633 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:31:56.811064 11099 connection.cc:535] client connection to 127.9.231.253:44633 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:31:56.811934 11099 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.9.231.253:44633)
W20260620 18:31:56.812804 11099 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.231.253:44633)
W20260620 18:31:56.813170 11099 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.9.231.253:44633)
W20260620 18:31:56.813910 11099 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.9.231.253:44633)
I20260620 18:31:57.011085 11575 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:31:57.011451 11575 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:57.012300 11575 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
W20260620 18:31:57.015030 11437 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:31:57.015307 11436 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
I20260620 18:31:57.015448 11436 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4587918520d849eaad95e35037e36608; no voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2
I20260620 18:31:57.015697 11575 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260620 18:31:57.211027 11572 raft_consensus.cc:482] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:57.211297 11572 raft_consensus.cc:504] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:57.212204 11572 leader_election.cc:288] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:57.212986 10751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdce1a08f6e34686a1a50676ada66941" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:31:57.213017 11041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdce1a08f6e34686a1a50676ada66941" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc" is_pre_election: true
I20260620 18:31:57.213305 10751 raft_consensus.cc:2454] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 0.
I20260620 18:31:57.213316 11041 raft_consensus.cc:2454] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 0.
I20260620 18:31:57.213907 10833 leader_election.cc:302] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:31:57.214349 11572 raft_consensus.cc:2781] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:31:57.214476 11572 raft_consensus.cc:482] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:31:57.214545 11572 raft_consensus.cc:3037] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:57.216766 11572 raft_consensus.cc:504] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:57.217396 11572 leader_election.cc:288] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Requested vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:57.218091 10751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdce1a08f6e34686a1a50676ada66941" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433"
I20260620 18:31:57.218091 11041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdce1a08f6e34686a1a50676ada66941" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc"
I20260620 18:31:57.218385 10751 raft_consensus.cc:3037] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:57.218394 11041 raft_consensus.cc:3037] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:31:57.221553 11041 raft_consensus.cc:2454] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
I20260620 18:31:57.221553 10751 raft_consensus.cc:2454] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
I20260620 18:31:57.222169 10833 leader_election.cc:302] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:31:57.222540 11572 raft_consensus.cc:2781] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:31:57.222715 11572 raft_consensus.cc:686] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 LEADER]: Becoming Leader. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Running, Role: LEADER
I20260620 18:31:57.223009 11572 consensus_queue.cc:237] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:31:57.695994 11587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:31:57.696223 11587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:57.697077 11587 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
W20260620 18:31:57.699949 11436 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
W20260620 18:31:57.700205 11437 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:31:57.700452 11437 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4587918520d849eaad95e35037e36608; no voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2
I20260620 18:31:57.700819 11587 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260620 18:31:58.277647 11586 consensus_queue.cc:1035] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:31:58.286563 11602 consensus_queue.cc:1035] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:31:58.348564 11606 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:31:58.348822 11606 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:58.349591 11606 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
W20260620 18:31:58.352509 11436 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
W20260620 18:31:58.352802 11437 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:31:58.352957 11437 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4587918520d849eaad95e35037e36608; no voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2
I20260620 18:31:58.353229 11606 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260620 18:31:58.804925 10143 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
/tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.254:43203
--webserver_interface=127.9.231.254
--webserver_port=45071
--builtin_ntp_servers=127.9.231.212:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260620 18:31:58.883710 11616 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:31:58.884057 11616 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:58.884829 11616 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
W20260620 18:31:58.887346 11437 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:31:58.888631 11436 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
I20260620 18:31:58.888845 11436 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4587918520d849eaad95e35037e36608; no voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2
I20260620 18:31:58.889284 11616 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260620 18:31:59.216833 11615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:31:59.217219 11615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:31:59.217362 11615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:31:59.225155 11615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:31:59.225315 11615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:31:59.225422 11615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:31:59.225535 11615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:31:59.237963 11615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.231.212:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.254:43203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.231.254
--webserver_port=45071
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 17:42:20 UTC on ccycloud.quasar-rymxqh.root.comops.site
build id 4457681
ASAN enabled
I20260620 18:31:59.239485 11615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:31:59.241510 11615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:31:59.255860 11634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:59.255970 11636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:31:59.255898 11633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:31:59.263010 11615 server_base.cc:1034] running on GCE node
I20260620 18:31:59.264118 11615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:31:59.266017 11615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:31:59.267390 11615 hybrid_clock.cc:648] HybridClock initialized: now 1781980319267303 us; error 60 us; skew 500 ppm
I20260620 18:31:59.267820 11615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 18:31:59.270790 11615 webserver.cc:460] Webserver started at http://127.9.231.254:45071/ using document root <none> and password file <none>
I20260620 18:31:59.271600 11615 fs_manager.cc:359] Metadata directory not provided
I20260620 18:31:59.271777 11615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:31:59.281536 11615 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.007s sys 0.000s
I20260620 18:31:59.288010 11643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:31:59.290609 11615 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.001s
I20260620 18:31:59.290959 11615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data,/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
uuid: "1cd216a992124f0fac2fb65754e6f7df"
format_stamp: "Formatted at 2026-06-20 18:31:45 on dist-test-slave-dv4w"
I20260620 18:31:59.293128 11615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:31:59.313467 11615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:31:59.314700 11615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:31:59.315090 11615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:31:59.339037 11615 rpc_server.cc:307] RPC server started. Bound to: 127.9.231.254:43203
I20260620 18:31:59.339066 11696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.231.254:43203 every 8 connection(s)
I20260620 18:31:59.341609 11615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data/info.pb
I20260620 18:31:59.349517 11697 sys_catalog.cc:263] Verifying existing consensus state
I20260620 18:31:59.352205 10143 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu as pid 11615
I20260620 18:31:59.353592 11697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Bootstrap starting.
I20260620 18:31:59.367264 11697 log.cc:824] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Log is configured to *not* fsync() on all Append() calls
I20260620 18:31:59.396626 11697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Bootstrap replayed 1/1 log segments. Stats: ops{read=26 overwritten=0 applied=26 ignored=0} inserts{seen=19 ignored=0} mutations{seen=14 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:31:59.397261 11697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Bootstrap complete.
I20260620 18:31:59.405809 11697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:59.406840 11697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cd216a992124f0fac2fb65754e6f7df, State: Initialized, Role: FOLLOWER
I20260620 18:31:59.407526 11697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 26, Last appended: 5.26, Last appended by leader: 26, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:59.408731 11701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:31:59.409410 11701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:31:59.409722 11697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:31:59.428586 11715 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:31:59.428727 11715 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: loading cluster ID for follower catalog manager: success
I20260620 18:31:59.434048 11715 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: acquiring CA information for follower catalog manager: success
I20260620 18:31:59.438364 11715 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:31:59.440440 11615 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:31:59.452450 11615 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:31:59.677417 11088 heartbeater.cc:347] Connected to a master server at 127.9.231.254:43203
I20260620 18:31:59.683179 11662 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" instance_seqno: 1781980309668499) as {username='slave'} at 127.9.231.195:50881; Asking this server to re-register.
I20260620 18:31:59.685161 11088 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:59.688792 11661 ts_manager.cc:194] Registered new tserver with Master: 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:31:59.795428 10945 heartbeater.cc:347] Connected to a master server at 127.9.231.254:43203
I20260620 18:31:59.799175 11661 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" instance_seqno: 1781980309047005) as {username='slave'} at 127.9.231.194:45455; Asking this server to re-register.
I20260620 18:31:59.800896 10945 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:59.803076 11661 ts_manager.cc:194] Registered new tserver with Master: a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:31:59.804952 10797 heartbeater.cc:347] Connected to a master server at 127.9.231.254:43203
I20260620 18:31:59.807287 11661 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" instance_seqno: 1781980308024586) as {username='slave'} at 127.9.231.193:46465; Asking this server to re-register.
I20260620 18:31:59.808157 10797 heartbeater.cc:464] Registering TS with master...
I20260620 18:31:59.810739 11661 ts_manager.cc:194] Registered new tserver with Master: 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961)
I20260620 18:31:59.995189 11725 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:31:59.995504 11725 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:31:59.997728 11725 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:31:59.999668 11648 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:00.001781 11648 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:32:00.007560 11462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1cd216a992124f0fac2fb65754e6f7df" candidate_term: 6 candidate_status { last_received { term: 5 index: 26 } } ignore_live_leader: false dest_uuid: "4587918520d849eaad95e35037e36608" is_pre_election: true
I20260620 18:32:00.008065 11462 raft_consensus.cc:2354] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1cd216a992124f0fac2fb65754e6f7df for earlier term 6. Current term is 7.
I20260620 18:32:00.009341 11648 leader_election.cc:398] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 6 pre-election: Vote denied by peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1cd216a992124f0fac2fb65754e6f7df for earlier term 6. Current term is 7.
I20260620 18:32:00.009487 11648 leader_election.cc:401] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260620 18:32:00.009991 11725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 5 FOLLOWER]: Advancing to term 7
I20260620 18:32:00.016079 11725 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1cd216a992124f0fac2fb65754e6f7df for earlier term 6. Current term is 7.
I20260620 18:32:00.378124 11416 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:32:00.378343 11416 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:32:00.646211 11737 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:00.646438 11737 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:00.647095 11737 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
I20260620 18:32:00.648926 11462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1cd216a992124f0fac2fb65754e6f7df" candidate_term: 8 candidate_status { last_received { term: 5 index: 26 } } ignore_live_leader: false dest_uuid: "4587918520d849eaad95e35037e36608" is_pre_election: true
I20260620 18:32:00.649391 11462 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1cd216a992124f0fac2fb65754e6f7df for term 8 because replica has last-logged OpId of term: 7 index: 35, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 26.
W20260620 18:32:00.650022 11648 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:32:00.650156 11648 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:32:00.650555 11737 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260620 18:32:01.261221 11749 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:32:01.261499 11749 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:01.262291 11749 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
I20260620 18:32:01.263414 11672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 8 candidate_status { last_received { term: 7 index: 35 } } ignore_live_leader: false dest_uuid: "1cd216a992124f0fac2fb65754e6f7df" is_pre_election: true
I20260620 18:32:01.263996 11672 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4587918520d849eaad95e35037e36608 in term 7.
I20260620 18:32:01.264693 11436 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608; no voters:
I20260620 18:32:01.265067 11749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260620 18:32:01.265173 11437 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:32:01.265240 11749 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:32:01.265340 11749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 7 FOLLOWER]: Advancing to term 8
I20260620 18:32:01.267910 11749 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:01.268707 11749 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 election: Requested vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
I20260620 18:32:01.269647 11672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 8 candidate_status { last_received { term: 7 index: 35 } } ignore_live_leader: false dest_uuid: "1cd216a992124f0fac2fb65754e6f7df"
I20260620 18:32:01.270356 11672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 7 FOLLOWER]: Advancing to term 8
W20260620 18:32:01.270579 11437 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:32:01.273631 11672 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4587918520d849eaad95e35037e36608 in term 8.
I20260620 18:32:01.274262 11436 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608; no voters: 650e0197335644718cd91d5018399ee2
I20260620 18:32:01.274768 11749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 8 FOLLOWER]: Leader election won for term 8
I20260620 18:32:01.275404 11749 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 8 LEADER]: Becoming Leader. State: Replica: 4587918520d849eaad95e35037e36608, State: Running, Role: LEADER
I20260620 18:32:01.276036 11749 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 7.35, Last appended by leader: 35, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:01.278275 11752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4587918520d849eaad95e35037e36608. Latest consensus state: current_term: 8 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:01.278786 11752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: LEADER
I20260620 18:32:01.279795 11754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:32:01.283241 11754 catalog_manager.cc:669] Loaded metadata for table default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]
I20260620 18:32:01.283833 11754 catalog_manager.cc:669] Loaded metadata for table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]
I20260620 18:32:01.284370 11754 catalog_manager.cc:669] Loaded metadata for table default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74]
I20260620 18:32:01.284788 11754 catalog_manager.cc:669] Loaded metadata for table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]
I20260620 18:32:01.285161 11754 catalog_manager.cc:669] Loaded metadata for table default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3]
I20260620 18:32:01.285538 11754 catalog_manager.cc:669] Loaded metadata for table default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59]
I20260620 18:32:01.285892 11754 catalog_manager.cc:669] Loaded metadata for table default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2]
I20260620 18:32:01.288467 11754 tablet_loader.cc:96] loaded metadata for tablet 1709fe2d11a24f52bd1812bc37e1714b (table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c])
I20260620 18:32:01.289280 11754 tablet_loader.cc:96] loaded metadata for tablet 4fb0f7d77bac4b11bc0aa4486b39aa97 (table default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047])
I20260620 18:32:01.289814 11754 tablet_loader.cc:96] loaded metadata for tablet 654f8832c4be4d9ea5d23b446696a63f (table default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59])
I20260620 18:32:01.290133 11754 tablet_loader.cc:96] loaded metadata for tablet 6a00bd448f6e4d1fbf99e4c11bb6e970 (table default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3])
I20260620 18:32:01.290622 11754 tablet_loader.cc:96] loaded metadata for tablet 8b82c74e9aa6451c8e7c537f7096aa25 (table default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2])
I20260620 18:32:01.290961 11754 tablet_loader.cc:96] loaded metadata for tablet acc2a8b335d14d20bad1719566a27f3d (table default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74])
I20260620 18:32:01.291561 11754 tablet_loader.cc:96] loaded metadata for tablet fdce1a08f6e34686a1a50676ada66941 (table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770])
I20260620 18:32:01.291915 11754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:32:01.293367 11754 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:32:01.293504 11754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:32:01.295219 11754 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:32:01.296474 11754 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Loaded TSK: 0
I20260620 18:32:01.297818 11754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:32:01.299316 11754 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260620 18:32:01.300813 11754 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260620 18:32:01.323005 11451 master_service.cc:946] ReplaceTablet: received request to replace tablet 4cdd0fd6fcb842239d0fb74758cd1290 from {username='slave'} at 127.0.0.1:39238
I20260620 18:32:01.323352 11450 master_service.cc:946] ReplaceTablet: received request to replace tablet b7a80fba1822404a98459c47faef4680 from {username='slave'} at 127.0.0.1:39238
I20260620 18:32:01.323833 11449 master_service.cc:946] ReplaceTablet: received request to replace tablet 1709fe2d11a24f52bd1812bc37e1714b from {username='slave'} at 127.0.0.1:39238
I20260620 18:32:01.323630 11452 master_service.cc:946] ReplaceTablet: received request to replace tablet b7a80fba1822404a98459c47faef4680 from {username='slave'} at 127.0.0.1:39238
I20260620 18:32:01.325783 11448 master_service.cc:946] ReplaceTablet: received request to replace tablet 8009d3fea30b4efa951e07d0bc2da1a5 from {username='slave'} at 127.0.0.1:39238
I20260620 18:32:01.328485 11450 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39238:
table { table_name: "default.Table_23d8d896ffe7406ca154449fd1a1eeab" } new_table_name: "default.Table_042b39ab8bd94475b7844eed01c35807" modify_external_catalogs: true
I20260620 18:32:01.329281 11451 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39238:
name: "default.Table_aa7e29fa2b8e4974a8ece0e46aba222a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:01.327910 11445 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39238:
name: "default.Table_1ba08641833a48838fae05c1f72333e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:01.328449 11447 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39238:
name: "default.Table_08795a50aec84f85bc35ae8025674185"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:01.328703 11446 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39238:
name: "default.Table_16d5e29e7ed8458180884abae667bf9d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:01.331104 11452 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39238:
table { table_name: "default.Table_be97363936284e2faa8400049925a98c" } new_table_name: "default.Table_4e545c1bf96b40de8e03778dfe0e4861" modify_external_catalogs: true
I20260620 18:32:01.331501 11444 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39238:
name: "default.Table_5854ef7ddc554da59e75ed6ff18718ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:01.331604 11443 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39238:
name: "default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:32:01.336109 11437 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 10 similar messages]
I20260620 18:32:01.336699 11672 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 8 FOLLOWER]: Refusing update from remote peer 4587918520d849eaad95e35037e36608: Log matching property violated. Preceding OpId in replica: term: 5 index: 26. Preceding OpId from leader: term: 8 index: 37. (index mismatch)
I20260620 18:32:01.331524 11448 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39238:
name: "default.Table_e001ba7ef7b44f09b6262dc3d55d0df0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:01.337610 11749 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 26, Time since last communication: 0.000s
W20260620 18:32:01.338043 11437 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 -> Peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Couldn't send request to peer 650e0197335644718cd91d5018399ee2. Status: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
F20260620 18:32:01.338884 11508 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_269d7d1648864d4e8b29b944b7115e6a
*** Check failure stack trace: ***
@ 0x7fb407eac9ad google::LogMessage::Fail() at ??:0
@ 0x7fb407eaf88a google::LogMessage::SendToLog() at ??:0
@ 0x7fb407eac4de google::LogMessage::Flush() at ??:0
@ 0x7fb407eac6d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7fb41da4aba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7fb41da49901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7fb41da5066c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7fb41da504e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb40aad6eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb40ddb06db start_thread at ??:0
I20260620 18:32:01.377063 11757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4587918520d849eaad95e35037e36608. Latest consensus state: current_term: 8 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:01.377485 11757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:01.377660 11759 mvcc.cc:205] Tried to move back new op lower bound from 7298991376391573504 to 7298991359280361472. Current Snapshot: MvccSnapshot[applied={T|T < 7298991376391573504}]
@ 0x7fb406d4661f clone at ??:0
I20260620 18:32:01.378041 11767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 8 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:01.378374 11767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:01.378854 11763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 8 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:01.379202 11763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:01.450093 10143 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu pid 11416
W20260620 18:32:01.600732 11099 connection.cc:535] client connection to 127.9.231.252:34267 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:01.601682 11099 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.9.231.252:34267)
W20260620 18:32:01.601866 11775 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:32:01.451219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.231.252:34267 (local address 127.0.0.1:44844)
0620 18:32:01.452011 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0620 18:32:01.452039 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0620 18:32:01.452222 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:32:01.601307 (+149085us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":371,"threads_started":1}
W20260620 18:32:01.601795 10832 connection.cc:535] client connection to 127.9.231.252:34267 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:01.602236 10977 connection.cc:535] client connection to 127.9.231.252:34267 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:01.602337 11099 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.9.231.252:34267)
W20260620 18:32:01.602646 11099 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.231.252:34267)
W20260620 18:32:01.602850 10143 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu pid 11416: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:01.603075 11099 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.9.231.252:34267)
I20260620 18:32:01.603137 10143 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu with pid 11416
I20260620 18:32:01.811282 10796 heartbeater.cc:503] Master 127.9.231.252:34267 was elected leader, sending a full tablet report...
I20260620 18:32:01.846514 11757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:32:01.846774 11757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:01.847545 11757 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:01.851099 11648 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:01.851368 11648 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:01.851605 11648 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:32:01.851989 11757 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260620 18:32:02.216609 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 6: this message will repeat every 5th retry.
I20260620 18:32:02.297209 11792 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:02.297497 11792 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:02.298328 11792 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:02.299912 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:32:02.300213 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:32:02.300382 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:32:02.300868 11792 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:32:02.502717 11793 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:32:02.503029 11793 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:02.503762 11793 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:02.506527 11648 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:02.506795 11648 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:02.506896 11648 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:32:02.507315 11793 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260620 18:32:02.589063 10143 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
/tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.252:34267
--webserver_interface=127.9.231.252
--webserver_port=42823
--builtin_ntp_servers=127.9.231.212:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260620 18:32:02.892773 11799 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:02.893080 11799 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:02.893855 11799 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:02.895203 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:32:02.895534 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:32:02.895653 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:32:02.895988 11799 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:32:02.963654 11797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:32:02.964077 11797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:32:02.964534 11797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:32:02.973860 11797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:32:02.973994 11797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:32:02.974038 11797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:32:02.974089 11797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:32:02.986243 11797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.231.212:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.252:34267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.231.252
--webserver_port=42823
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 17:42:20 UTC on ccycloud.quasar-rymxqh.root.comops.site
build id 4457681
ASAN enabled
I20260620 18:32:02.987715 11797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:32:02.989663 11797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:32:03.004945 11805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:32:03.004896 11807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:32:03.004878 11804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:32:03.006664 11797 server_base.cc:1034] running on GCE node
I20260620 18:32:03.007956 11797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:32:03.010083 11797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:32:03.011624 11797 hybrid_clock.cc:648] HybridClock initialized: now 1781980323011510 us; error 98 us; skew 500 ppm
I20260620 18:32:03.012126 11797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 18:32:03.015919 11797 webserver.cc:460] Webserver started at http://127.9.231.252:42823/ using document root <none> and password file <none>
I20260620 18:32:03.016979 11797 fs_manager.cc:359] Metadata directory not provided
I20260620 18:32:03.017171 11797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:32:03.023722 11797 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 18:32:03.027878 11814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:32:03.030207 11797 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260620 18:32:03.030508 11797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data,/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/wal
uuid: "4587918520d849eaad95e35037e36608"
format_stamp: "Formatted at 2026-06-20 18:31:46 on dist-test-slave-dv4w"
I20260620 18:32:03.031780 11797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:32:03.065589 11797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:32:03.067416 11797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:32:03.067788 11797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:32:03.093736 11797 rpc_server.cc:307] RPC server started. Bound to: 127.9.231.252:34267
I20260620 18:32:03.093886 11867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.231.252:34267 every 8 connection(s)
I20260620 18:32:03.095590 11797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-2/data/info.pb
I20260620 18:32:03.104151 11868 sys_catalog.cc:263] Verifying existing consensus state
I20260620 18:32:03.104718 10143 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu as pid 11797
I20260620 18:32:03.109874 11868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Bootstrap starting.
I20260620 18:32:03.132089 11868 log.cc:824] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Log is configured to *not* fsync() on all Append() calls
I20260620 18:32:03.137712 10944 heartbeater.cc:347] Connected to a master server at 127.9.231.252:34267
W20260620 18:32:03.157392 10977 consensus_peers.cc:487] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5. This is attempt 6: this message will repeat every 5th retry.
I20260620 18:32:03.158500 11087 heartbeater.cc:347] Connected to a master server at 127.9.231.252:34267
I20260620 18:32:03.172611 11868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Bootstrap replayed 1/1 log segments. Stats: ops{read=38 overwritten=1 applied=35 ignored=0} inserts{seen=23 ignored=0} mutations{seen=22 ignored=0} orphaned_commits=0. Pending: 2 replicates
I20260620 18:32:03.173493 11868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Bootstrap complete.
I20260620 18:32:03.182755 11868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 8 FOLLOWER]: Replica starting. Triggering 2 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:03.187023 11868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4587918520d849eaad95e35037e36608, State: Initialized, Role: FOLLOWER
I20260620 18:32:03.187960 11868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 35, Last appended: 8.37, Last appended by leader: 37, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:03.188941 11880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:03.189365 11880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:03.190610 11868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:32:03.216315 11894 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:32:03.216522 11894 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: loading cluster ID for follower catalog manager: success
I20260620 18:32:03.221084 11894 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: acquiring CA information for follower catalog manager: success
I20260620 18:32:03.232764 11894 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:32:03.239626 11797 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:32:03.247094 11797 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:32:03.327610 10796 heartbeater.cc:347] Connected to a master server at 127.9.231.252:34267
I20260620 18:32:03.327994 10796 heartbeater.cc:503] Master 127.9.231.252:34267 was elected leader, sending a full tablet report...
I20260620 18:32:03.332995 11833 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" instance_seqno: 1781980308024586) as {username='slave'} at 127.9.231.193:36391; Asking this server to re-register.
I20260620 18:32:03.334065 10796 heartbeater.cc:464] Registering TS with master...
I20260620 18:32:03.336483 11833 ts_manager.cc:194] Registered new tserver with Master: 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961)
I20260620 18:32:03.443369 11899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:32:03.443734 11899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:03.444739 11899 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:03.450021 11648 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:32:03.455854 11843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1cd216a992124f0fac2fb65754e6f7df" candidate_term: 9 candidate_status { last_received { term: 8 index: 37 } } ignore_live_leader: false dest_uuid: "4587918520d849eaad95e35037e36608" is_pre_election: true
I20260620 18:32:03.456764 11843 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cd216a992124f0fac2fb65754e6f7df in term 8.
I20260620 18:32:03.457628 11648 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608; no voters: 650e0197335644718cd91d5018399ee2
I20260620 18:32:03.458063 11899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260620 18:32:03.458244 11899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:32:03.458366 11899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 8 FOLLOWER]: Advancing to term 9
I20260620 18:32:03.461779 11899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:03.462985 11899 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 9 election: Requested vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
I20260620 18:32:03.463930 11843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1cd216a992124f0fac2fb65754e6f7df" candidate_term: 9 candidate_status { last_received { term: 8 index: 37 } } ignore_live_leader: false dest_uuid: "4587918520d849eaad95e35037e36608"
I20260620 18:32:03.464229 11843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 8 FOLLOWER]: Advancing to term 9
W20260620 18:32:03.464609 11648 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:32:03.470248 11843 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cd216a992124f0fac2fb65754e6f7df in term 9.
I20260620 18:32:03.471686 11648 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608; no voters: 650e0197335644718cd91d5018399ee2
I20260620 18:32:03.472167 11899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 9 FOLLOWER]: Leader election won for term 9
I20260620 18:32:03.472831 11899 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 9 LEADER]: Becoming Leader. State: Replica: 1cd216a992124f0fac2fb65754e6f7df, State: Running, Role: LEADER
I20260620 18:32:03.473426 11899 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 8.37, Last appended by leader: 37, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:03.475593 11903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cd216a992124f0fac2fb65754e6f7df. Latest consensus state: current_term: 9 leader_uuid: "1cd216a992124f0fac2fb65754e6f7df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:03.475997 11903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: LEADER
I20260620 18:32:03.503007 11833 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" instance_seqno: 1781980309047005) as {username='slave'} at 127.9.231.194:50317; Asking this server to re-register.
I20260620 18:32:03.503844 10944 heartbeater.cc:464] Registering TS with master...
I20260620 18:32:03.505928 11833 ts_manager.cc:194] Registered new tserver with Master: a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:32:03.662822 11833 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" instance_seqno: 1781980309668499) as {username='slave'} at 127.9.231.195:55857; Asking this server to re-register.
I20260620 18:32:03.663774 11087 heartbeater.cc:464] Registering TS with master...
I20260620 18:32:03.665788 11833 ts_manager.cc:194] Registered new tserver with Master: 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:03.862672 11648 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df -> Peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Couldn't send request to peer 650e0197335644718cd91d5018399ee2. Status: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 18:32:04.050088 11908 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:04.050400 11908 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:04.052840 11908 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
I20260620 18:32:04.053757 11672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 10 candidate_status { last_received { term: 8 index: 37 } } ignore_live_leader: false dest_uuid: "1cd216a992124f0fac2fb65754e6f7df" is_pre_election: true
W20260620 18:32:04.054250 11817 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:04.056188 11817 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:32:04.056480 11817 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4587918520d849eaad95e35037e36608; no voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2
I20260620 18:32:04.056977 11908 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260620 18:32:04.076938 11843 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Refusing update from remote peer 1cd216a992124f0fac2fb65754e6f7df: Log matching property violated. Preceding OpId in replica: term: 8 index: 37. Preceding OpId from leader: term: 9 index: 38. (index mismatch)
I20260620 18:32:04.078332 11903 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [LEADER]: Connected to new peer: Peer: permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 35, Time since last communication: 0.000s
I20260620 18:32:04.084540 11908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cd216a992124f0fac2fb65754e6f7df. Latest consensus state: current_term: 9 leader_uuid: "1cd216a992124f0fac2fb65754e6f7df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:04.084874 11908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:04.089154 11903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "1cd216a992124f0fac2fb65754e6f7df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:04.089550 11903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: LEADER
I20260620 18:32:04.089351 11914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 9 leader_uuid: "1cd216a992124f0fac2fb65754e6f7df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:04.089764 11914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: LEADER
I20260620 18:32:04.091403 11908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "1cd216a992124f0fac2fb65754e6f7df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:04.091742 11908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:04.091403 11921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "1cd216a992124f0fac2fb65754e6f7df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:04.091990 11921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:04.122606 11905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:32:04.125411 11905 catalog_manager.cc:669] Loaded metadata for table default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]
I20260620 18:32:04.125928 11905 catalog_manager.cc:669] Loaded metadata for table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]
I20260620 18:32:04.126295 11905 catalog_manager.cc:669] Loaded metadata for table default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74]
I20260620 18:32:04.126663 11905 catalog_manager.cc:669] Loaded metadata for table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]
I20260620 18:32:04.126997 11905 catalog_manager.cc:669] Loaded metadata for table default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3]
I20260620 18:32:04.127383 11905 catalog_manager.cc:669] Loaded metadata for table default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59]
I20260620 18:32:04.127733 11905 catalog_manager.cc:669] Loaded metadata for table default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2]
I20260620 18:32:04.130645 11905 tablet_loader.cc:96] loaded metadata for tablet 297d7a2d0f1347a397adf2933daae869 (table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c])
I20260620 18:32:04.131286 11905 tablet_loader.cc:96] loaded metadata for tablet 4fb0f7d77bac4b11bc0aa4486b39aa97 (table default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047])
I20260620 18:32:04.131875 11905 tablet_loader.cc:96] loaded metadata for tablet 654f8832c4be4d9ea5d23b446696a63f (table default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59])
I20260620 18:32:04.132231 11905 tablet_loader.cc:96] loaded metadata for tablet 6a00bd448f6e4d1fbf99e4c11bb6e970 (table default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3])
I20260620 18:32:04.132887 11905 tablet_loader.cc:96] loaded metadata for tablet 8b82c74e9aa6451c8e7c537f7096aa25 (table default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2])
I20260620 18:32:04.133241 11905 tablet_loader.cc:96] loaded metadata for tablet acc2a8b335d14d20bad1719566a27f3d (table default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74])
I20260620 18:32:04.134196 11905 tablet_loader.cc:96] loaded metadata for tablet fdce1a08f6e34686a1a50676ada66941 (table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770])
I20260620 18:32:04.134676 11905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:32:04.136240 11905 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:32:04.136497 11905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:32:04.138312 11905 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:32:04.139482 11905 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Loaded TSK: 0
I20260620 18:32:04.140935 11905 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:32:04.142012 11905 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260620 18:32:04.143003 11905 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260620 18:32:04.162945 11088 heartbeater.cc:503] Master 127.9.231.254:43203 was elected leader, sending a full tablet report...
I20260620 18:32:04.167050 11662 catalog_manager.cc:5495] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc reported cstate change: term changed from 0 to 1, leader changed from <none> to a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194). New cstate: current_term: 1 leader_uuid: "a0ae894b0758446aa6219aaa5267d1b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } }
I20260620 18:32:04.167865 11662 catalog_manager.cc:5495] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195). New cstate: current_term: 1 leader_uuid: "9a4ce671d3984e1a8592bc29d45612bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } health_report { overall_health: HEALTHY } } }
I20260620 18:32:04.168545 11662 catalog_manager.cc:5495] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc reported cstate change: term changed from 0 to 1, leader changed from <none> to a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194). New cstate: current_term: 1 leader_uuid: "a0ae894b0758446aa6219aaa5267d1b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } }
I20260620 18:32:04.179286 11661 master_service.cc:946] ReplaceTablet: received request to replace tablet fdce1a08f6e34686a1a50676ada66941 from {username='slave'} at 127.0.0.1:51752
I20260620 18:32:04.181506 11659 master_service.cc:946] ReplaceTablet: received request to replace tablet 4fb0f7d77bac4b11bc0aa4486b39aa97 from {username='slave'} at 127.0.0.1:51752
I20260620 18:32:04.181828 11658 master_service.cc:946] ReplaceTablet: received request to replace tablet 4cdd0fd6fcb842239d0fb74758cd1290 from {username='slave'} at 127.0.0.1:51752
I20260620 18:32:04.181591 11660 master_service.cc:946] ReplaceTablet: received request to replace tablet b7a80fba1822404a98459c47faef4680 from {username='slave'} at 127.0.0.1:51752
I20260620 18:32:04.185304 11655 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51752:
name: "default.Table_fce67ec520194c7fabdca1fce51af8a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:04.186508 11658 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51752:
table { table_name: "default.Table_f27c96a5d11944f6a75123e42b5b2ce5" } modify_external_catalogs: true
I20260620 18:32:04.187830 11658 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51752:
table { table_name: "default.Table_f27c96a5d11944f6a75123e42b5b2ce5" } modify_external_catalogs: true
I20260620 18:32:04.188333 11654 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51752:
table { table_name: "default.Table_88afa4cb37564293a220876026ee1931" } modify_external_catalogs: true
I20260620 18:32:04.185318 11660 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51752:
name: "default.Table_e1ad292ef5e24a628dcf2d699e127e57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:04.187381 11657 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51752:
table { table_name: "default.Table_9e9a3aec7a67407ba81fe1f6797b55f6" } new_table_name: "default.Table_70f8c2fd535343bfb5649c75f207510a" modify_external_catalogs: true
I20260620 18:32:04.189605 11654 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51752:
table { table_name: "default.Table_88afa4cb37564293a220876026ee1931" } modify_external_catalogs: true
I20260620 18:32:04.187505 11656 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51752:
name: "default.Table_e4c5c6935f7a48cebd0f828771b1fe01"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:04.187875 11653 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51752:
name: "default.Table_08795a50aec84f85bc35ae8025674185"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:04.192873 11661 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Sending DeleteTablet for 3 replicas of tablet fdce1a08f6e34686a1a50676ada66941
I20260620 18:32:04.194406 11661 catalog_manager.cc:6565] ReplaceTablet: tablet fdce1a08f6e34686a1a50676ada66941 deleted and replaced by tablet ac236b968be243ffa51a59f03798498f
I20260620 18:32:04.200790 11661 master_service.cc:946] ReplaceTablet: received request to replace tablet 1709fe2d11a24f52bd1812bc37e1714b from {username='slave'} at 127.0.0.1:51752
I20260620 18:32:04.202965 11661 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51752:
name: "default.Table_16d5e29e7ed8458180884abae667bf9d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260620 18:32:04.207536 11714 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_269d7d1648864d4e8b29b944b7115e6a
*** Check failure stack trace: ***
I20260620 18:32:04.208679 11662 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51752:
name: "default.Table_aa7e29fa2b8e4974a8ece0e46aba222a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7fc25489f9ad google::LogMessage::Fail() at ??:0
@ 0x7fc2548a288a google::LogMessage::SendToLog() at ??:0
@ 0x7fc25489f4de google::LogMessage::Flush() at ??:0
@ 0x7fc25489f6d1 google::LogMessage::~LogMessage() at ??:0
I20260620 18:32:04.222313 11020 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cdd0fd6fcb842239d0fb74758cd1290 with delete_type TABLET_DATA_DELETED (replaced by tablet 8b82c74e9aa6451c8e7c537f7096aa25) from {username='slave'} at 127.0.0.1:38104
I20260620 18:32:04.222448 11021 tablet_service.cc:1514] Processing DeleteTablet for tablet 1709fe2d11a24f52bd1812bc37e1714b with delete_type TABLET_DATA_DELETED (replaced by tablet 297d7a2d0f1347a397adf2933daae869) from {username='slave'} at 127.0.0.1:38104
I20260620 18:32:04.222644 11019 tablet_service.cc:1514] Processing DeleteTablet for tablet 8009d3fea30b4efa951e07d0bc2da1a5 with delete_type TABLET_DATA_DELETED (replaced by tablet acc2a8b335d14d20bad1719566a27f3d) from {username='slave'} at 127.0.0.1:38104
I20260620 18:32:04.222918 11022 tablet_service.cc:1514] Processing DeleteTablet for tablet fdce1a08f6e34686a1a50676ada66941 with delete_type TABLET_DATA_DELETED (replaced by tablet ac236b968be243ffa51a59f03798498f) from {username='slave'} at 127.0.0.1:38104
I20260620 18:32:04.222863 11018 tablet_service.cc:1514] Processing DeleteTablet for tablet b7a80fba1822404a98459c47faef4680 with delete_type TABLET_DATA_DELETED (replaced by tablet 6a00bd448f6e4d1fbf99e4c11bb6e970) from {username='slave'} at 127.0.0.1:38104
@ 0x7fc26a43dba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260620 18:32:04.226945 11659 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Sending DeleteTablet for 3 replicas of tablet 4fb0f7d77bac4b11bc0aa4486b39aa97
I20260620 18:32:04.227626 11659 catalog_manager.cc:6565] ReplaceTablet: tablet 4fb0f7d77bac4b11bc0aa4486b39aa97 deleted and replaced by tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e
I20260620 18:32:04.229189 11018 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fb0f7d77bac4b11bc0aa4486b39aa97 with delete_type TABLET_DATA_DELETED (replaced by tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e) from {username='slave'} at 127.0.0.1:38104
I20260620 18:32:04.228113 11659 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51752:
table { table_name: "default.Table_23d8d896ffe7406ca154449fd1a1eeab" } new_table_name: "default.Table_042b39ab8bd94475b7844eed01c35807" modify_external_catalogs: true
I20260620 18:32:04.235232 11018 tablet_service.cc:1467] Processing CreateTablet for tablet ac236b968be243ffa51a59f03798498f (DEFAULT_TABLE table=default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]), partition=RANGE (key) PARTITION UNBOUNDED
@ 0x7fc26a43c901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260620 18:32:04.235965 11018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac236b968be243ffa51a59f03798498f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:04.236677 11933 tablet_replica.cc:330] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc: stopping tablet replica
I20260620 18:32:04.237026 11933 raft_consensus.cc:2229] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:32:04.237546 11933 raft_consensus.cc:2258] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:04.238500 11933 ts_tablet_manager.cc:1905] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:04.241215 11021 tablet_service.cc:1467] Processing CreateTablet for tablet 297d7a2d0f1347a397adf2933daae869 (DEFAULT_TABLE table=default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:04.241729 11021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 297d7a2d0f1347a397adf2933daae869. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:04.242449 10879 tablet_service.cc:1514] Processing DeleteTablet for tablet fdce1a08f6e34686a1a50676ada66941 with delete_type TABLET_DATA_DELETED (replaced by tablet ac236b968be243ffa51a59f03798498f) from {username='slave'} at 127.0.0.1:53408
I20260620 18:32:04.242875 10878 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fb0f7d77bac4b11bc0aa4486b39aa97 with delete_type TABLET_DATA_DELETED (replaced by tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e) from {username='slave'} at 127.0.0.1:53408
I20260620 18:32:04.243404 10877 tablet_service.cc:1467] Processing CreateTablet for tablet ac236b968be243ffa51a59f03798498f (DEFAULT_TABLE table=default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:04.243800 10877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac236b968be243ffa51a59f03798498f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:04.246137 10876 tablet_service.cc:1467] Processing CreateTablet for tablet 297d7a2d0f1347a397adf2933daae869 (DEFAULT_TABLE table=default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:04.246553 10876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 297d7a2d0f1347a397adf2933daae869. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fc26a44366c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260620 18:32:04.249912 11022 tablet_service.cc:1467] Processing CreateTablet for tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e (DEFAULT_TABLE table=default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:04.250352 11022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:04.254412 11933 ts_tablet_manager.cc:1918] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:04.254577 11933 log.cc:1196] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal/wals/1709fe2d11a24f52bd1812bc37e1714b
I20260620 18:32:04.255074 11933 ts_tablet_manager.cc:1939] T 1709fe2d11a24f52bd1812bc37e1714b P 9a4ce671d3984e1a8592bc29d45612bc: Deleting consensus metadata
I20260620 18:32:04.257200 10878 tablet_service.cc:1467] Processing CreateTablet for tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e (DEFAULT_TABLE table=default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:04.257794 10878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260620 18:32:04.258296 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 1: this message will repeat every 5th retry.
@ 0x7fc26a4434e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260620 18:32:04.261296 11649 catalog_manager.cc:4841] TS 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): tablet 1709fe2d11a24f52bd1812bc37e1714b (table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]) successfully deleted
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260620 18:32:04.267802 11933 tablet_replica.cc:330] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc: stopping tablet replica
I20260620 18:32:04.268165 11933 raft_consensus.cc:2229] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:32:04.268468 11933 raft_consensus.cc:2258] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:04.269317 11933 ts_tablet_manager.cc:1905] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:04.271885 11937 tablet_bootstrap.cc:492] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
@ 0x7fc2574c9eaf kudu::Thread::SuperviseThread() at ??:0
I20260620 18:32:04.275146 11936 tablet_bootstrap.cc:492] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
@ 0x7fc25a7a36db start_thread at ??:0
@ 0x7fc25373961f clone at ??:0
I20260620 18:32:04.277794 10728 tablet_service.cc:1514] Processing DeleteTablet for tablet fdce1a08f6e34686a1a50676ada66941 with delete_type TABLET_DATA_DELETED (replaced by tablet ac236b968be243ffa51a59f03798498f) from {username='slave'} at 127.0.0.1:49734
I20260620 18:32:04.278081 10730 tablet_service.cc:1467] Processing CreateTablet for tablet 297d7a2d0f1347a397adf2933daae869 (DEFAULT_TABLE table=default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:04.278288 10727 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fb0f7d77bac4b11bc0aa4486b39aa97 with delete_type TABLET_DATA_DELETED (replaced by tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e) from {username='slave'} at 127.0.0.1:49734
I20260620 18:32:04.278697 10730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 297d7a2d0f1347a397adf2933daae869. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:04.279294 10729 tablet_service.cc:1467] Processing CreateTablet for tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e (DEFAULT_TABLE table=default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:04.280095 10729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:04.277827 10731 tablet_service.cc:1467] Processing CreateTablet for tablet ac236b968be243ffa51a59f03798498f (DEFAULT_TABLE table=default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:04.281190 10731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac236b968be243ffa51a59f03798498f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:04.284237 11933 ts_tablet_manager.cc:1918] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:04.285130 11933 log.cc:1196] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal/wals/4cdd0fd6fcb842239d0fb74758cd1290
I20260620 18:32:04.285691 11933 ts_tablet_manager.cc:1939] T 4cdd0fd6fcb842239d0fb74758cd1290 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting consensus metadata
I20260620 18:32:04.287163 11933 tablet_replica.cc:330] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc: stopping tablet replica
I20260620 18:32:04.287410 11933 raft_consensus.cc:2229] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:32:04.288026 11933 raft_consensus.cc:2258] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:04.294309 11935 tablet_replica.cc:330] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4: stopping tablet replica
I20260620 18:32:04.294649 11935 raft_consensus.cc:2229] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:32:04.295235 11936 tablet_bootstrap.cc:654] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:04.295565 11935 raft_consensus.cc:2258] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:04.296531 11935 ts_tablet_manager.cc:1905] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:04.297046 11933 ts_tablet_manager.cc:1905] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:04.306972 11937 tablet_bootstrap.cc:654] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:04.312955 11939 tablet_replica.cc:330] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433: stopping tablet replica
I20260620 18:32:04.313225 11939 raft_consensus.cc:2229] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:32:04.313560 11939 raft_consensus.cc:2258] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:04.314149 11937 tablet_bootstrap.cc:492] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:32:04.314343 11937 ts_tablet_manager.cc:1397] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.043s user 0.005s sys 0.000s
I20260620 18:32:04.317606 11937 raft_consensus.cc:348] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.318001 11937 raft_consensus.cc:374] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:04.318121 11937 raft_consensus.cc:729] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:32:04.318527 11937 consensus_queue.cc:260] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.320986 11939 ts_tablet_manager.cc:1905] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:04.321100 11937 ts_tablet_manager.cc:1428] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260620 18:32:04.321468 11937 tablet_bootstrap.cc:492] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
I20260620 18:32:04.323299 11937 tablet_bootstrap.cc:654] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:04.325155 11933 ts_tablet_manager.cc:1918] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:04.325348 11933 log.cc:1196] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal/wals/8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:32:04.325878 11933 ts_tablet_manager.cc:1939] T 8009d3fea30b4efa951e07d0bc2da1a5 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting consensus metadata
I20260620 18:32:04.327268 11933 tablet_replica.cc:330] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc: stopping tablet replica
I20260620 18:32:04.327522 11933 raft_consensus.cc:2229] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:32:04.327754 11933 raft_consensus.cc:2258] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:04.329730 11941 tablet_bootstrap.cc:492] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:32:04.329936 11936 tablet_bootstrap.cc:492] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:32:04.330101 11936 ts_tablet_manager.cc:1397] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.055s user 0.009s sys 0.007s
I20260620 18:32:04.331149 11936 raft_consensus.cc:348] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.331501 11936 raft_consensus.cc:374] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:04.331605 11936 raft_consensus.cc:729] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:32:04.332443 11941 tablet_bootstrap.cc:654] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:04.332593 11936 consensus_queue.cc:260] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.334214 11923 raft_consensus.cc:482] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:04.334421 11923 raft_consensus.cc:504] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.334890 11936 ts_tablet_manager.cc:1428] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260620 18:32:04.335168 11923 leader_election.cc:288] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:32:04.335286 11936 tablet_bootstrap.cc:492] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
I20260620 18:32:04.336092 11939 ts_tablet_manager.cc:1918] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:04.336243 11939 log.cc:1196] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal/wals/fdce1a08f6e34686a1a50676ada66941
I20260620 18:32:04.337594 11939 ts_tablet_manager.cc:1939] T fdce1a08f6e34686a1a50676ada66941 P 7696506dcc9746d0aea9b1129a3e9433: Deleting consensus metadata
I20260620 18:32:04.337924 10751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "297d7a2d0f1347a397adf2933daae869" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
W20260620 18:32:04.338708 10832 leader_election.cc:341] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Illegal state: must be running to vote when last-logged opid is not known
I20260620 18:32:04.339102 11937 tablet_bootstrap.cc:492] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:32:04.339280 11937 ts_tablet_manager.cc:1397] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
I20260620 18:32:04.339422 11935 ts_tablet_manager.cc:1918] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:04.339550 11935 log.cc:1196] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/wal/wals/fdce1a08f6e34686a1a50676ada66941
I20260620 18:32:04.340034 11935 ts_tablet_manager.cc:1939] T fdce1a08f6e34686a1a50676ada66941 P a0ae894b0758446aa6219aaa5267d1b4: Deleting consensus metadata
I20260620 18:32:04.340196 11937 raft_consensus.cc:348] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.340430 11936 tablet_bootstrap.cc:654] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:04.340479 11937 raft_consensus.cc:374] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:04.340565 11937 raft_consensus.cc:729] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:32:04.340780 11937 consensus_queue.cc:260] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.340955 11041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "297d7a2d0f1347a397adf2933daae869" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc" is_pre_election: true
I20260620 18:32:04.341189 11939 tablet_replica.cc:330] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433: stopping tablet replica
I20260620 18:32:04.341406 11939 raft_consensus.cc:2229] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:32:04.341598 11939 raft_consensus.cc:2258] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:04.341868 11933 ts_tablet_manager.cc:1905] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:04.342319 11937 ts_tablet_manager.cc:1428] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 18:32:04.342617 11937 tablet_bootstrap.cc:492] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
W20260620 18:32:04.343628 10833 leader_election.cc:341] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Illegal state: must be running to vote when last-logged opid is not known
I20260620 18:32:04.343766 10833 leader_election.cc:302] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:32:04.345485 11923 raft_consensus.cc:2726] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:32:04.347987 11937 tablet_bootstrap.cc:654] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:04.348878 11939 ts_tablet_manager.cc:1905] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:04.351733 11937 tablet_bootstrap.cc:492] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:32:04.351874 11937 ts_tablet_manager.cc:1397] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.002s
I20260620 18:32:04.352680 11937 raft_consensus.cc:348] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.352927 11937 raft_consensus.cc:374] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:04.353022 11937 raft_consensus.cc:729] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:32:04.355046 11933 ts_tablet_manager.cc:1918] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:04.355273 11933 log.cc:1196] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal/wals/fdce1a08f6e34686a1a50676ada66941
I20260620 18:32:04.355149 11937 consensus_queue.cc:260] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.355757 11933 ts_tablet_manager.cc:1939] T fdce1a08f6e34686a1a50676ada66941 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting consensus metadata
I20260620 18:32:04.356355 11941 tablet_bootstrap.cc:492] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:32:04.356449 11935 tablet_replica.cc:330] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4: stopping tablet replica
I20260620 18:32:04.356508 11941 ts_tablet_manager.cc:1397] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.027s user 0.007s sys 0.000s
I20260620 18:32:04.356659 11935 raft_consensus.cc:2229] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:32:04.357375 11941 raft_consensus.cc:348] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.357635 11941 raft_consensus.cc:374] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:04.357726 11941 raft_consensus.cc:729] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:32:04.358409 11936 tablet_bootstrap.cc:492] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:32:04.358237 11941 consensus_queue.cc:260] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.358606 11936 ts_tablet_manager.cc:1397] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.004s
I20260620 18:32:04.359890 11933 tablet_replica.cc:330] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc: stopping tablet replica
I20260620 18:32:04.360136 11933 raft_consensus.cc:2229] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260620 18:32:04.360219 11935 raft_consensus.cc:2258] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:04.360387 11933 raft_consensus.cc:2258] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:04.361078 11933 ts_tablet_manager.cc:1905] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:04.361331 11935 ts_tablet_manager.cc:1905] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:04.359747 11936 raft_consensus.cc:348] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.362970 11936 raft_consensus.cc:374] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:04.363070 11936 raft_consensus.cc:729] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:32:04.363348 11936 consensus_queue.cc:260] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.363981 11936 ts_tablet_manager.cc:1428] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260620 18:32:04.364344 11936 tablet_bootstrap.cc:492] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
I20260620 18:32:04.366276 11936 tablet_bootstrap.cc:654] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:04.367507 11937 ts_tablet_manager.cc:1428] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.016s user 0.000s sys 0.003s
I20260620 18:32:04.367637 11939 ts_tablet_manager.cc:1918] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:04.367816 11939 log.cc:1196] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal/wals/4fb0f7d77bac4b11bc0aa4486b39aa97
I20260620 18:32:04.368575 11939 ts_tablet_manager.cc:1939] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 7696506dcc9746d0aea9b1129a3e9433: Deleting consensus metadata
I20260620 18:32:04.371626 11935 ts_tablet_manager.cc:1918] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:04.371824 11935 log.cc:1196] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/wal/wals/4fb0f7d77bac4b11bc0aa4486b39aa97
I20260620 18:32:04.372331 11935 ts_tablet_manager.cc:1939] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P a0ae894b0758446aa6219aaa5267d1b4: Deleting consensus metadata
I20260620 18:32:04.373874 11941 ts_tablet_manager.cc:1428] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.017s user 0.003s sys 0.000s
I20260620 18:32:04.374912 11941 tablet_bootstrap.cc:492] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:32:04.376053 11933 ts_tablet_manager.cc:1918] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260620 18:32:04.376215 11933 log.cc:1196] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal/wals/b7a80fba1822404a98459c47faef4680
I20260620 18:32:04.376847 11933 ts_tablet_manager.cc:1939] T b7a80fba1822404a98459c47faef4680 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting consensus metadata
I20260620 18:32:04.377154 11941 tablet_bootstrap.cc:654] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:04.378974 11933 tablet_replica.cc:330] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc: stopping tablet replica
I20260620 18:32:04.379271 11933 raft_consensus.cc:2229] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:32:04.379516 11933 raft_consensus.cc:2258] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:04.380265 11933 ts_tablet_manager.cc:1905] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:04.380616 11936 tablet_bootstrap.cc:492] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:32:04.380846 11936 ts_tablet_manager.cc:1397] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.003s
I20260620 18:32:04.382081 11936 raft_consensus.cc:348] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.382514 11936 raft_consensus.cc:374] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:04.382663 11936 raft_consensus.cc:729] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:32:04.383023 11936 consensus_queue.cc:260] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.383814 11936 ts_tablet_manager.cc:1428] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260620 18:32:04.385775 11941 tablet_bootstrap.cc:492] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:32:04.385953 11941 ts_tablet_manager.cc:1397] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.008s
I20260620 18:32:04.386713 11941 raft_consensus.cc:348] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.386955 11941 raft_consensus.cc:374] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:04.387028 11941 raft_consensus.cc:729] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:32:04.387352 11941 consensus_queue.cc:260] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.387890 11941 ts_tablet_manager.cc:1428] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 18:32:04.390656 11941 tablet_bootstrap.cc:492] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:32:04.393366 11941 tablet_bootstrap.cc:654] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:04.397176 11933 ts_tablet_manager.cc:1918] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:04.397404 11933 log.cc:1196] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal/wals/4fb0f7d77bac4b11bc0aa4486b39aa97
I20260620 18:32:04.398114 11933 ts_tablet_manager.cc:1939] T 4fb0f7d77bac4b11bc0aa4486b39aa97 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting consensus metadata
I20260620 18:32:04.403062 11941 tablet_bootstrap.cc:492] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:32:04.403275 11941 ts_tablet_manager.cc:1397] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.001s
I20260620 18:32:04.404518 11941 raft_consensus.cc:348] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.404863 11941 raft_consensus.cc:374] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:04.404965 11941 raft_consensus.cc:729] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:32:04.405299 11941 consensus_queue.cc:260] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.405997 11941 ts_tablet_manager.cc:1428] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260620 18:32:04.454948 11923 raft_consensus.cc:482] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:04.455209 11923 raft_consensus.cc:504] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.456394 11923 leader_election.cc:288] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:32:04.457198 10751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac236b968be243ffa51a59f03798498f" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:32:04.457160 11041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac236b968be243ffa51a59f03798498f" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc" is_pre_election: true
I20260620 18:32:04.457585 10751 raft_consensus.cc:2454] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 0.
I20260620 18:32:04.457592 11041 raft_consensus.cc:2454] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 0.
I20260620 18:32:04.458215 10833 leader_election.cc:302] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:32:04.458568 11923 raft_consensus.cc:2781] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:32:04.458709 11923 raft_consensus.cc:482] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:32:04.458839 11923 raft_consensus.cc:3037] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:04.460928 11923 raft_consensus.cc:504] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.461669 11923 leader_election.cc:288] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Requested vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:32:04.462333 11041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac236b968be243ffa51a59f03798498f" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc"
I20260620 18:32:04.462482 10751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac236b968be243ffa51a59f03798498f" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433"
I20260620 18:32:04.462625 11041 raft_consensus.cc:3037] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:04.462742 10751 raft_consensus.cc:3037] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:04.465883 11041 raft_consensus.cc:2454] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
I20260620 18:32:04.465883 10751 raft_consensus.cc:2454] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
I20260620 18:32:04.466528 10832 leader_election.cc:302] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:32:04.466549 11925 raft_consensus.cc:482] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:04.466758 11925 raft_consensus.cc:504] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.467088 11923 raft_consensus.cc:2781] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:32:04.467406 11923 raft_consensus.cc:686] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [term 1 LEADER]: Becoming Leader. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Running, Role: LEADER
I20260620 18:32:04.467638 11925 leader_election.cc:288] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:32:04.468372 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "297d7a2d0f1347a397adf2933daae869" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4" is_pre_election: true
I20260620 18:32:04.468417 11923 consensus_queue.cc:237] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.468660 10899 raft_consensus.cc:2454] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 0.
I20260620 18:32:04.469360 10978 leader_election.cc:302] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:32:04.469633 11925 raft_consensus.cc:2781] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:32:04.469741 11925 raft_consensus.cc:482] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:32:04.469660 10751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "297d7a2d0f1347a397adf2933daae869" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:32:04.469818 11925 raft_consensus.cc:3037] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:04.469920 10751 raft_consensus.cc:2454] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 0.
I20260620 18:32:04.472018 11925 raft_consensus.cc:504] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.472693 11925 leader_election.cc:288] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 election: Requested vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:32:04.473470 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "297d7a2d0f1347a397adf2933daae869" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4"
I20260620 18:32:04.473745 10899 raft_consensus.cc:3037] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:04.476065 10751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "297d7a2d0f1347a397adf2933daae869" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433"
I20260620 18:32:04.476363 10751 raft_consensus.cc:3037] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:04.477782 10899 raft_consensus.cc:2454] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 1.
I20260620 18:32:04.478441 10978 leader_election.cc:302] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:32:04.478775 11925 raft_consensus.cc:2781] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:32:04.478897 11925 raft_consensus.cc:686] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 LEADER]: Becoming Leader. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Running, Role: LEADER
I20260620 18:32:04.479203 10751 raft_consensus.cc:2454] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 1.
I20260620 18:32:04.479202 11925 consensus_queue.cc:237] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.561378 11923 raft_consensus.cc:482] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:04.561668 11923 raft_consensus.cc:504] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.562579 11923 leader_election.cc:288] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:32:04.563421 10751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360d15e9cf9d4d2aa21dfe7ae0c5c65e" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:32:04.563483 11041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360d15e9cf9d4d2aa21dfe7ae0c5c65e" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc" is_pre_election: true
I20260620 18:32:04.563750 10751 raft_consensus.cc:2454] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 0.
I20260620 18:32:04.563830 11041 raft_consensus.cc:2454] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 0.
I20260620 18:32:04.564316 10832 leader_election.cc:302] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:32:04.564824 11923 raft_consensus.cc:2781] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:32:04.564955 11923 raft_consensus.cc:482] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:32:04.565064 11923 raft_consensus.cc:3037] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:04.568519 11923 raft_consensus.cc:504] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.569386 11923 leader_election.cc:288] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Requested vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:04.569933 10976 connection.cc:535] server connection from 127.0.0.1:38104 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:04.569994 10684 connection.cc:535] server connection from 127.0.0.1:49734 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 18:32:04.570235 11041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360d15e9cf9d4d2aa21dfe7ae0c5c65e" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc"
W20260620 18:32:04.570386 10976 connection.cc:535] client connection to 127.9.231.254:43203 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:04.570386 10685 connection.cc:535] client connection to 127.9.231.254:43203 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 18:32:04.570564 11041 raft_consensus.cc:3037] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
W20260620 18:32:04.571357 11098 connection.cc:535] client connection to 127.9.231.254:43203 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:04.572127 11098 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.9.231.254:43203)
W20260620 18:32:04.572870 11098 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.9.231.254:43203)
W20260620 18:32:04.573199 11098 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.231.254:43203)
W20260620 18:32:04.573540 10831 connection.cc:535] client connection to 127.9.231.254:43203 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:04.574016 11098 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.9.231.254:43203)
W20260620 18:32:04.574873 10832 connection.cc:535] server connection from 127.0.0.1:53408 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 18:32:04.575261 10751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360d15e9cf9d4d2aa21dfe7ae0c5c65e" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433"
I20260620 18:32:04.575546 10751 raft_consensus.cc:3037] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:04.576140 11041 raft_consensus.cc:2454] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
I20260620 18:32:04.576804 10833 leader_election.cc:302] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:32:04.577208 11923 raft_consensus.cc:2781] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:32:04.577417 11923 raft_consensus.cc:686] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [term 1 LEADER]: Becoming Leader. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Running, Role: LEADER
I20260620 18:32:04.577711 11923 consensus_queue.cc:237] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:04.579298 10751 raft_consensus.cc:2454] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
W20260620 18:32:04.640666 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 1: this message will repeat every 5th retry.
I20260620 18:32:04.754074 11965 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1cd216a992124f0fac2fb65754e6f7df)
I20260620 18:32:04.754391 11965 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:04.755175 11965 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
W20260620 18:32:04.757860 11817 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:04.758112 11816 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
I20260620 18:32:04.758225 11816 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4587918520d849eaad95e35037e36608; no voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2
I20260620 18:32:04.758463 11965 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260620 18:32:04.818892 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 1: this message will repeat every 5th retry.
I20260620 18:32:05.364617 11925 consensus_queue.cc:1035] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:32:05.372763 11974 consensus_queue.cc:1035] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:32:05.455674 11980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1cd216a992124f0fac2fb65754e6f7df)
I20260620 18:32:05.456036 11980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:05.457032 11980 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
W20260620 18:32:05.460459 11816 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
W20260620 18:32:05.460793 11817 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:32:05.460940 11817 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4587918520d849eaad95e35037e36608; no voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2
I20260620 18:32:05.461386 11980 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260620 18:32:05.560369 11733 consensus_queue.cc:1035] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:32:05.567023 11922 consensus_queue.cc:1035] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:32:05.644078 11922 consensus_queue.cc:1035] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:32:05.652701 11922 consensus_queue.cc:1035] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:32:06.422127 10143 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
/tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.254:43203
--webserver_interface=127.9.231.254
--webserver_port=45071
--builtin_ntp_servers=127.9.231.212:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260620 18:32:06.489305 11997 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1cd216a992124f0fac2fb65754e6f7df)
I20260620 18:32:06.489593 11997 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:06.490408 11997 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
W20260620 18:32:06.493076 11817 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:06.493361 11816 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
I20260620 18:32:06.493584 11816 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4587918520d849eaad95e35037e36608; no voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2
I20260620 18:32:06.493897 11997 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260620 18:32:06.718334 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 6: this message will repeat every 5th retry.
W20260620 18:32:06.849241 11996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:32:06.849663 11996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:32:06.849809 11996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:32:06.859128 11996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:32:06.859294 11996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:32:06.859376 11996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:32:06.859470 11996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:32:06.871474 11996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.231.212:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.254:43203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.231.254
--webserver_port=45071
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 17:42:20 UTC on ccycloud.quasar-rymxqh.root.comops.site
build id 4457681
ASAN enabled
I20260620 18:32:06.873528 11996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:32:06.875869 11996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:32:06.891579 12011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:32:06.891670 12012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:32:06.891546 12014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:32:06.892947 11996 server_base.cc:1034] running on GCE node
I20260620 18:32:06.894136 11996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:32:06.896178 11996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:32:06.897550 11996 hybrid_clock.cc:648] HybridClock initialized: now 1781980326897448 us; error 54 us; skew 500 ppm
I20260620 18:32:06.898052 11996 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 18:32:06.901417 11996 webserver.cc:460] Webserver started at http://127.9.231.254:45071/ using document root <none> and password file <none>
I20260620 18:32:06.902439 11996 fs_manager.cc:359] Metadata directory not provided
I20260620 18:32:06.902618 11996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:32:06.909889 11996 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 18:32:06.914438 12020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:32:06.916687 11996 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260620 18:32:06.916944 11996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data,/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
uuid: "1cd216a992124f0fac2fb65754e6f7df"
format_stamp: "Formatted at 2026-06-20 18:31:45 on dist-test-slave-dv4w"
I20260620 18:32:06.918154 11996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:32:06.935578 11996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:32:06.936877 11996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:32:06.937212 11996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:32:06.963140 11996 rpc_server.cc:307] RPC server started. Bound to: 127.9.231.254:43203
I20260620 18:32:06.963158 12071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.231.254:43203 every 8 connection(s)
I20260620 18:32:06.965545 11996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data/info.pb
I20260620 18:32:06.969743 10143 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu as pid 11996
I20260620 18:32:06.977639 12072 sys_catalog.cc:263] Verifying existing consensus state
I20260620 18:32:06.984653 12072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Bootstrap starting.
I20260620 18:32:07.002933 12072 log.cc:824] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Log is configured to *not* fsync() on all Append() calls
I20260620 18:32:07.034614 11088 heartbeater.cc:347] Connected to a master server at 127.9.231.254:43203
I20260620 18:32:07.046511 12072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Bootstrap replayed 1/1 log segments. Stats: ops{read=43 overwritten=0 applied=43 ignored=0} inserts{seen=26 ignored=0} mutations{seen=31 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:32:07.047243 12072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Bootstrap complete.
I20260620 18:32:07.060281 12072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 9 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:07.062667 12072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cd216a992124f0fac2fb65754e6f7df, State: Initialized, Role: FOLLOWER
I20260620 18:32:07.063807 12072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 43, Last appended: 9.43, Last appended by leader: 43, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:07.065748 10945 heartbeater.cc:347] Connected to a master server at 127.9.231.254:43203
I20260620 18:32:07.065362 12080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 9 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:07.065902 12080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:07.066805 12072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:32:07.088665 12095 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:32:07.088932 12095 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: loading cluster ID for follower catalog manager: success
I20260620 18:32:07.094449 12095 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: acquiring CA information for follower catalog manager: success
I20260620 18:32:07.098462 12095 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:32:07.098922 11996 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:32:07.110836 11996 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:32:07.161935 11733 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:32:07.162209 11733 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:07.164067 11733 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:07.166182 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:32:07.166652 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:32:07.166949 10833 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:32:07.167642 11733 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:32:07.176541 10797 heartbeater.cc:347] Connected to a master server at 127.9.231.254:43203
I20260620 18:32:07.179775 12037 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" instance_seqno: 1781980308024586) as {username='slave'} at 127.9.231.193:52587; Asking this server to re-register.
I20260620 18:32:07.180673 10797 heartbeater.cc:464] Registering TS with master...
I20260620 18:32:07.182641 12037 ts_manager.cc:194] Registered new tserver with Master: 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961)
I20260620 18:32:07.203050 12100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1cd216a992124f0fac2fb65754e6f7df)
I20260620 18:32:07.203245 12100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:07.204025 12100 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
W20260620 18:32:07.208765 11817 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:32:07.215607 12047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 10 candidate_status { last_received { term: 9 index: 43 } } ignore_live_leader: false dest_uuid: "1cd216a992124f0fac2fb65754e6f7df" is_pre_election: true
I20260620 18:32:07.216557 12047 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4587918520d849eaad95e35037e36608 in term 9.
I20260620 18:32:07.217337 11816 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608; no voters: 650e0197335644718cd91d5018399ee2
I20260620 18:32:07.217778 12100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260620 18:32:07.217936 12100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1cd216a992124f0fac2fb65754e6f7df)
I20260620 18:32:07.218061 12100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 9 FOLLOWER]: Advancing to term 10
I20260620 18:32:07.220216 12100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:07.220997 12100 leader_election.cc:288] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 election: Requested vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633)
I20260620 18:32:07.221818 12047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4587918520d849eaad95e35037e36608" candidate_term: 10 candidate_status { last_received { term: 9 index: 43 } } ignore_live_leader: false dest_uuid: "1cd216a992124f0fac2fb65754e6f7df"
I20260620 18:32:07.222080 12047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 9 FOLLOWER]: Advancing to term 10
W20260620 18:32:07.226107 11817 leader_election.cc:334] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:32:07.228574 12047 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4587918520d849eaad95e35037e36608 in term 10.
I20260620 18:32:07.229291 11816 leader_election.cc:302] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608; no voters: 650e0197335644718cd91d5018399ee2
I20260620 18:32:07.229708 12100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 10 FOLLOWER]: Leader election won for term 10
I20260620 18:32:07.230433 12100 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [term 10 LEADER]: Becoming Leader. State: Replica: 4587918520d849eaad95e35037e36608, State: Running, Role: LEADER
I20260620 18:32:07.231151 12100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 9.43, Last appended by leader: 43, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:07.233659 12104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4587918520d849eaad95e35037e36608. Latest consensus state: current_term: 10 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:07.234124 12104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: LEADER
I20260620 18:32:07.234935 12106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:32:07.238333 12106 catalog_manager.cc:669] Loaded metadata for table default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]
I20260620 18:32:07.239248 12106 catalog_manager.cc:669] Loaded metadata for table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]
I20260620 18:32:07.239738 12106 catalog_manager.cc:669] Loaded metadata for table default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74]
I20260620 18:32:07.240113 12106 catalog_manager.cc:669] Loaded metadata for table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]
I20260620 18:32:07.240890 12106 catalog_manager.cc:669] Loaded metadata for table default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3]
I20260620 18:32:07.241364 12106 catalog_manager.cc:669] Loaded metadata for table default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59]
I20260620 18:32:07.241715 12106 catalog_manager.cc:669] Loaded metadata for table default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2]
I20260620 18:32:07.243995 12106 tablet_loader.cc:96] loaded metadata for tablet 297d7a2d0f1347a397adf2933daae869 (table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c])
I20260620 18:32:07.244405 12106 tablet_loader.cc:96] loaded metadata for tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e (table default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047])
I20260620 18:32:07.245168 12106 tablet_loader.cc:96] loaded metadata for tablet 654f8832c4be4d9ea5d23b446696a63f (table default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59])
I20260620 18:32:07.245512 12106 tablet_loader.cc:96] loaded metadata for tablet 6a00bd448f6e4d1fbf99e4c11bb6e970 (table default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3])
I20260620 18:32:07.245980 12106 tablet_loader.cc:96] loaded metadata for tablet 8b82c74e9aa6451c8e7c537f7096aa25 (table default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2])
I20260620 18:32:07.246287 12106 tablet_loader.cc:96] loaded metadata for tablet ac236b968be243ffa51a59f03798498f (table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770])
I20260620 18:32:07.246582 12106 tablet_loader.cc:96] loaded metadata for tablet acc2a8b335d14d20bad1719566a27f3d (table default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74])
I20260620 18:32:07.247268 12106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:32:07.248450 12106 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:32:07.248523 12106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:32:07.249903 12106 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:32:07.250945 12106 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Loaded TSK: 0
I20260620 18:32:07.251920 12106 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:32:07.252977 12106 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260620 18:32:07.253922 12106 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260620 18:32:07.293967 11832 master_service.cc:946] ReplaceTablet: received request to replace tablet 297d7a2d0f1347a397adf2933daae869 from {username='slave'} at 127.0.0.1:45314
I20260620 18:32:07.294190 11831 master_service.cc:946] ReplaceTablet: received request to replace tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e from {username='slave'} at 127.0.0.1:45314
I20260620 18:32:07.295033 11828 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45314:
table { table_name: "default.Table_269d7d1648864d4e8b29b944b7115e6a" } new_table_name: "default.Table_45bc016e4a9b4288a42f33020e656b17" modify_external_catalogs: true
I20260620 18:32:07.296180 11830 master_service.cc:946] ReplaceTablet: received request to replace tablet b7a80fba1822404a98459c47faef4680 from {username='slave'} at 127.0.0.1:45314
I20260620 18:32:07.296674 11824 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45314:
table { table_name: "default.Table_be97363936284e2faa8400049925a98c" } new_table_name: "default.Table_4e545c1bf96b40de8e03778dfe0e4861" modify_external_catalogs: true
I20260620 18:32:07.297688 11830 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45314:
name: "default.Table_5854ef7ddc554da59e75ed6ff18718ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:07.298623 11829 master_service.cc:946] ReplaceTablet: received request to replace tablet ac236b968be243ffa51a59f03798498f from {username='slave'} at 127.0.0.1:45314
I20260620 18:32:07.296289 11826 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45314:
name: "default.Table_e001ba7ef7b44f09b6262dc3d55d0df0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:07.300746 11827 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45314:
table { table_name: "default.Table_009e06deb1f84f74b4a32287a8c44b5a" } new_table_name: "default.Table_7105b83c587d4cf0a6add4272b0defc6" modify_external_catalogs: true
I20260620 18:32:07.293751 11833 master_service.cc:946] ReplaceTablet: received request to replace tablet 297d7a2d0f1347a397adf2933daae869 from {username='slave'} at 127.0.0.1:45314
I20260620 18:32:07.301991 11825 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45314:
name: "default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:07.307396 12047 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Refusing update from remote peer 4587918520d849eaad95e35037e36608: Log matching property violated. Preceding OpId in replica: term: 9 index: 43. Preceding OpId from leader: term: 10 index: 45. (index mismatch)
I20260620 18:32:07.308730 12104 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 43, Time since last communication: 0.000s
F20260620 18:32:07.315986 11893 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_269d7d1648864d4e8b29b944b7115e6a
*** Check failure stack trace: ***
@ 0x7fef78f939ad google::LogMessage::Fail() at ??:0
@ 0x7fef78f9688a google::LogMessage::SendToLog() at ??:0
I20260620 18:32:07.321463 12080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4587918520d849eaad95e35037e36608. Latest consensus state: current_term: 10 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:07.321837 12080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7fef78f934de google::LogMessage::Flush() at ??:0
I20260620 18:32:07.323900 12107 mvcc.cc:205] Tried to move back new op lower bound from 7298991420640481280 to 7298991420344549376. Current Snapshot: MvccSnapshot[applied={T|T < 7298991420640481280}]
@ 0x7fef78f936d1 google::LogMessage::~LogMessage() at ??:0
I20260620 18:32:07.325496 12108 mvcc.cc:205] Tried to move back new op lower bound from 7298991420640481280 to 7298991420344549376. Current Snapshot: MvccSnapshot[applied={T|T < 7298991420640481280}]
W20260620 18:32:07.325907 11817 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 -> Peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Couldn't send request to peer 650e0197335644718cd91d5018399ee2. Status: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 18:32:07.335981 12080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 10 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:07.336405 12080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:07.338716 11832 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Sending DeleteTablet for 3 replicas of tablet 297d7a2d0f1347a397adf2933daae869
I20260620 18:32:07.340276 11829 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Sending DeleteTablet for 3 replicas of tablet ac236b968be243ffa51a59f03798498f
I20260620 18:32:07.340829 11832 catalog_manager.cc:6565] ReplaceTablet: tablet 297d7a2d0f1347a397adf2933daae869 deleted and replaced by tablet 13425835965446758595ff9d8101dd06
I20260620 18:32:07.340853 11829 catalog_manager.cc:6565] ReplaceTablet: tablet ac236b968be243ffa51a59f03798498f deleted and replaced by tablet 1a9ec7bf576448abb541eda4936306b2
I20260620 18:32:07.365263 11018 tablet_service.cc:1514] Processing DeleteTablet for tablet 297d7a2d0f1347a397adf2933daae869 with delete_type TABLET_DATA_DELETED (replaced by tablet 13425835965446758595ff9d8101dd06) from {username='slave'} at 127.0.0.1:38108
I20260620 18:32:07.365368 11021 tablet_service.cc:1514] Processing DeleteTablet for tablet ac236b968be243ffa51a59f03798498f with delete_type TABLET_DATA_DELETED (replaced by tablet 1a9ec7bf576448abb541eda4936306b2) from {username='slave'} at 127.0.0.1:38108
I20260620 18:32:07.366726 11831 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608: Sending DeleteTablet for 3 replicas of tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e
I20260620 18:32:07.367290 11831 catalog_manager.cc:6565] ReplaceTablet: tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e deleted and replaced by tablet 1ec6b4c37ff249d3971fe75f54ddeae2
I20260620 18:32:07.369284 11021 tablet_service.cc:1514] Processing DeleteTablet for tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e with delete_type TABLET_DATA_DELETED (replaced by tablet 1ec6b4c37ff249d3971fe75f54ddeae2) from {username='slave'} at 127.0.0.1:38108
I20260620 18:32:07.370718 10731 tablet_service.cc:1514] Processing DeleteTablet for tablet 297d7a2d0f1347a397adf2933daae869 with delete_type TABLET_DATA_DELETED (replaced by tablet 13425835965446758595ff9d8101dd06) from {username='slave'} at 127.0.0.1:49736
I20260620 18:32:07.370882 10729 tablet_service.cc:1514] Processing DeleteTablet for tablet ac236b968be243ffa51a59f03798498f with delete_type TABLET_DATA_DELETED (replaced by tablet 1a9ec7bf576448abb541eda4936306b2) from {username='slave'} at 127.0.0.1:49736
I20260620 18:32:07.371163 10730 tablet_service.cc:1514] Processing DeleteTablet for tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e with delete_type TABLET_DATA_DELETED (replaced by tablet 1ec6b4c37ff249d3971fe75f54ddeae2) from {username='slave'} at 127.0.0.1:49736
I20260620 18:32:07.371556 12122 tablet_replica.cc:330] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc: stopping tablet replica
I20260620 18:32:07.371795 12122 raft_consensus.cc:2229] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:32:07.372272 12122 raft_consensus.cc:2258] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:07.373075 12122 ts_tablet_manager.cc:1905] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7fef8eb31ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260620 18:32:07.387161 12122 ts_tablet_manager.cc:1918] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:07.387344 12122 log.cc:1196] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal/wals/297d7a2d0f1347a397adf2933daae869
I20260620 18:32:07.387873 12122 ts_tablet_manager.cc:1939] T 297d7a2d0f1347a397adf2933daae869 P 9a4ce671d3984e1a8592bc29d45612bc: Deleting consensus metadata
I20260620 18:32:07.389277 11829 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45314:
name: "default.Table_fce67ec520194c7fabdca1fce51af8a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:07.390113 12123 tablet_replica.cc:330] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433: stopping tablet replica
I20260620 18:32:07.390214 11832 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45314:
table { table_name: "default.Table_9e9a3aec7a67407ba81fe1f6797b55f6" } new_table_name: "default.Table_70f8c2fd535343bfb5649c75f207510a" modify_external_catalogs: true
I20260620 18:32:07.390413 12123 raft_consensus.cc:2229] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:32:07.390187 11833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45314:
name: "default.Table_e4c5c6935f7a48cebd0f828771b1fe01"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:07.390728 12123 raft_consensus.cc:2258] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:07.384188 11831 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45314:
table { table_name: "default.Table_88afa4cb37564293a220876026ee1931" } modify_external_catalogs: true
I20260620 18:32:07.391109 11831 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45314:
table { table_name: "default.Table_88afa4cb37564293a220876026ee1931" } modify_external_catalogs: true
I20260620 18:32:07.391547 12123 ts_tablet_manager.cc:1905] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:07.393288 12100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 10 leader_uuid: "4587918520d849eaad95e35037e36608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:07.393628 12100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4587918520d849eaad95e35037e36608 [sys.catalog]: This master's current role is: LEADER
I20260620 18:32:07.397253 12037 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" instance_seqno: 1781980309047005) as {username='slave'} at 127.9.231.194:34417; Asking this server to re-register.
I20260620 18:32:07.398133 11818 catalog_manager.cc:4841] TS 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): tablet 297d7a2d0f1347a397adf2933daae869 (table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]) successfully deleted
I20260620 18:32:07.398731 10945 heartbeater.cc:464] Registering TS with master...
I20260620 18:32:07.398751 12122 tablet_replica.cc:330] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc: stopping tablet replica
I20260620 18:32:07.399011 12122 raft_consensus.cc:2229] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:32:07.399240 12122 raft_consensus.cc:2258] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:07.400704 12037 ts_manager.cc:194] Registered new tserver with Master: a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:32:07.400730 11018 tablet_service.cc:1467] Processing CreateTablet for tablet 1a9ec7bf576448abb541eda4936306b2 (DEFAULT_TABLE table=default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:07.400950 10731 tablet_service.cc:1467] Processing CreateTablet for tablet 1a9ec7bf576448abb541eda4936306b2 (DEFAULT_TABLE table=default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:07.401317 11018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a9ec7bf576448abb541eda4936306b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:07.401499 10731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a9ec7bf576448abb541eda4936306b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:07.405234 10730 tablet_service.cc:1467] Processing CreateTablet for tablet 1ec6b4c37ff249d3971fe75f54ddeae2 (DEFAULT_TABLE table=default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:07.405941 10730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ec6b4c37ff249d3971fe75f54ddeae2. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fef8eb30901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260620 18:32:07.408902 10729 tablet_service.cc:1467] Processing CreateTablet for tablet 13425835965446758595ff9d8101dd06 (DEFAULT_TABLE table=default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:07.409602 10729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13425835965446758595ff9d8101dd06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:07.414305 11021 tablet_service.cc:1467] Processing CreateTablet for tablet 1ec6b4c37ff249d3971fe75f54ddeae2 (DEFAULT_TABLE table=default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:07.414542 10879 tablet_service.cc:1514] Processing DeleteTablet for tablet 297d7a2d0f1347a397adf2933daae869 with delete_type TABLET_DATA_DELETED (replaced by tablet 13425835965446758595ff9d8101dd06) from {username='slave'} at 127.0.0.1:53422
I20260620 18:32:07.414901 11021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ec6b4c37ff249d3971fe75f54ddeae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:07.415098 10876 tablet_service.cc:1514] Processing DeleteTablet for tablet ac236b968be243ffa51a59f03798498f with delete_type TABLET_DATA_DELETED (replaced by tablet 1a9ec7bf576448abb541eda4936306b2) from {username='slave'} at 127.0.0.1:53422
I20260620 18:32:07.415374 10878 tablet_service.cc:1514] Processing DeleteTablet for tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e with delete_type TABLET_DATA_DELETED (replaced by tablet 1ec6b4c37ff249d3971fe75f54ddeae2) from {username='slave'} at 127.0.0.1:53422
I20260620 18:32:07.416114 11022 tablet_service.cc:1467] Processing CreateTablet for tablet 13425835965446758595ff9d8101dd06 (DEFAULT_TABLE table=default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:07.416752 11022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13425835965446758595ff9d8101dd06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:07.421934 10877 tablet_service.cc:1467] Processing CreateTablet for tablet 13425835965446758595ff9d8101dd06 (DEFAULT_TABLE table=default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:07.422380 10876 tablet_service.cc:1467] Processing CreateTablet for tablet 1ec6b4c37ff249d3971fe75f54ddeae2 (DEFAULT_TABLE table=default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:07.422487 10877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13425835965446758595ff9d8101dd06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:07.422715 10876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ec6b4c37ff249d3971fe75f54ddeae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:07.421917 10878 tablet_service.cc:1467] Processing CreateTablet for tablet 1a9ec7bf576448abb541eda4936306b2 (DEFAULT_TABLE table=default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:07.423015 10878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a9ec7bf576448abb541eda4936306b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:07.435977 12128 tablet_bootstrap.cc:492] T 1a9ec7bf576448abb541eda4936306b2 P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
@ 0x7fef8eb3766c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260620 18:32:07.438126 12123 ts_tablet_manager.cc:1918] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:07.438316 12123 log.cc:1196] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal/wals/297d7a2d0f1347a397adf2933daae869
I20260620 18:32:07.439046 12123 ts_tablet_manager.cc:1939] T 297d7a2d0f1347a397adf2933daae869 P 7696506dcc9746d0aea9b1129a3e9433: Deleting consensus metadata
I20260620 18:32:07.440596 12128 tablet_bootstrap.cc:654] T 1a9ec7bf576448abb541eda4936306b2 P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:07.441023 11817 catalog_manager.cc:4841] TS 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): tablet 297d7a2d0f1347a397adf2933daae869 (table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]) successfully deleted
I20260620 18:32:07.441291 12122 ts_tablet_manager.cc:1905] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:07.441493 12123 tablet_replica.cc:330] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433: stopping tablet replica
I20260620 18:32:07.441758 12123 raft_consensus.cc:2229] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:32:07.442008 12123 raft_consensus.cc:2258] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:07.442862 12123 ts_tablet_manager.cc:1905] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:07.451581 12122 ts_tablet_manager.cc:1918] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:07.451810 12122 log.cc:1196] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal/wals/ac236b968be243ffa51a59f03798498f
I20260620 18:32:07.452430 12123 ts_tablet_manager.cc:1918] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:07.452631 12123 log.cc:1196] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal/wals/ac236b968be243ffa51a59f03798498f
I20260620 18:32:07.452968 12128 tablet_bootstrap.cc:492] T 1a9ec7bf576448abb541eda4936306b2 P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:32:07.453090 12123 ts_tablet_manager.cc:1939] T ac236b968be243ffa51a59f03798498f P 7696506dcc9746d0aea9b1129a3e9433: Deleting consensus metadata
I20260620 18:32:07.453168 12128 ts_tablet_manager.cc:1397] T 1a9ec7bf576448abb541eda4936306b2 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.004s
I20260620 18:32:07.454185 12128 raft_consensus.cc:348] T 1a9ec7bf576448abb541eda4936306b2 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.454506 12128 raft_consensus.cc:374] T 1a9ec7bf576448abb541eda4936306b2 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:07.454610 12128 raft_consensus.cc:729] T 1a9ec7bf576448abb541eda4936306b2 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:32:07.454594 11817 catalog_manager.cc:4841] TS 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): tablet ac236b968be243ffa51a59f03798498f (table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]) successfully deleted
I20260620 18:32:07.455112 12123 tablet_replica.cc:330] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433: stopping tablet replica
I20260620 18:32:07.454929 12128 consensus_queue.cc:260] T 1a9ec7bf576448abb541eda4936306b2 P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.455338 12123 raft_consensus.cc:2229] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:32:07.455641 12128 ts_tablet_manager.cc:1428] T 1a9ec7bf576448abb541eda4936306b2 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:32:07.456000 12128 tablet_bootstrap.cc:492] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
I20260620 18:32:07.456455 12127 tablet_replica.cc:330] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4: stopping tablet replica
I20260620 18:32:07.456494 12123 raft_consensus.cc:2258] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:07.456727 12127 raft_consensus.cc:2229] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:32:07.456991 12127 raft_consensus.cc:2258] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:07.457548 12123 ts_tablet_manager.cc:1905] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:07.457765 12127 ts_tablet_manager.cc:1905] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:07.457790 12037 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" instance_seqno: 1781980309668499) as {username='slave'} at 127.9.231.195:58675; Asking this server to re-register.
I20260620 18:32:07.459347 11088 heartbeater.cc:464] Registering TS with master...
I20260620 18:32:07.461380 12037 ts_manager.cc:194] Registered new tserver with Master: 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
@ 0x7fef8eb374e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260620 18:32:07.462610 12131 tablet_bootstrap.cc:492] T 1a9ec7bf576448abb541eda4936306b2 P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
I20260620 18:32:07.467226 12123 ts_tablet_manager.cc:1918] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:07.467420 12123 log.cc:1196] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal/wals/360d15e9cf9d4d2aa21dfe7ae0c5c65e
I20260620 18:32:07.467782 12131 tablet_bootstrap.cc:654] T 1a9ec7bf576448abb541eda4936306b2 P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:07.468144 12123 ts_tablet_manager.cc:1939] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 7696506dcc9746d0aea9b1129a3e9433: Deleting consensus metadata
I20260620 18:32:07.470530 11817 catalog_manager.cc:4841] TS 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): tablet 360d15e9cf9d4d2aa21dfe7ae0c5c65e (table default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]) successfully deleted
I20260620 18:32:07.471339 12131 tablet_bootstrap.cc:492] T 1a9ec7bf576448abb541eda4936306b2 P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:32:07.471573 12131 ts_tablet_manager.cc:1397] T 1a9ec7bf576448abb541eda4936306b2 P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260620 18:32:07.472754 12126 tablet_bootstrap.cc:492] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260620 18:32:07.473047 12131 raft_consensus.cc:348] T 1a9ec7bf576448abb541eda4936306b2 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.473449 12131 raft_consensus.cc:374] T 1a9ec7bf576448abb541eda4936306b2 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:07.473646 12131 raft_consensus.cc:729] T 1a9ec7bf576448abb541eda4936306b2 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:32:07.474094 12131 consensus_queue.cc:260] T 1a9ec7bf576448abb541eda4936306b2 P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.475911 12126 tablet_bootstrap.cc:654] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:07.476486 12131 ts_tablet_manager.cc:1428] T 1a9ec7bf576448abb541eda4936306b2 P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260620 18:32:07.476843 12131 tablet_bootstrap.cc:492] T 13425835965446758595ff9d8101dd06 P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
@ 0x7fef7bbbdeaf kudu::Thread::SuperviseThread() at ??:0
I20260620 18:32:07.479842 12131 tablet_bootstrap.cc:654] T 13425835965446758595ff9d8101dd06 P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:07.481191 12127 ts_tablet_manager.cc:1918] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:07.481351 12122 ts_tablet_manager.cc:1939] T ac236b968be243ffa51a59f03798498f P 9a4ce671d3984e1a8592bc29d45612bc: Deleting consensus metadata
I20260620 18:32:07.481389 12127 log.cc:1196] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/wal/wals/297d7a2d0f1347a397adf2933daae869
I20260620 18:32:07.481856 12128 tablet_bootstrap.cc:654] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:07.481885 12127 ts_tablet_manager.cc:1939] T 297d7a2d0f1347a397adf2933daae869 P a0ae894b0758446aa6219aaa5267d1b4: Deleting consensus metadata
I20260620 18:32:07.482805 12122 tablet_replica.cc:330] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc: stopping tablet replica
I20260620 18:32:07.483031 12122 raft_consensus.cc:2229] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:32:07.483069 11818 catalog_manager.cc:4841] TS a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): tablet 297d7a2d0f1347a397adf2933daae869 (table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]) successfully deleted
I20260620 18:32:07.483251 12122 raft_consensus.cc:2258] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:07.483928 12122 ts_tablet_manager.cc:1905] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:07.483964 11818 catalog_manager.cc:4841] TS 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): tablet ac236b968be243ffa51a59f03798498f (table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]) successfully deleted
W20260620 18:32:07.485155 10832 consensus_peers.cc:487] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360d15e9cf9d4d2aa21dfe7ae0c5c65e. This is attempt 1: this message will repeat every 5th retry.
I20260620 18:32:07.486131 12127 tablet_replica.cc:330] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4: stopping tablet replica
I20260620 18:32:07.486369 12127 raft_consensus.cc:2229] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:32:07.486730 12127 raft_consensus.cc:2258] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:07.487354 12127 ts_tablet_manager.cc:1905] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:07.489768 12131 tablet_bootstrap.cc:492] T 13425835965446758595ff9d8101dd06 P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:32:07.490029 12131 ts_tablet_manager.cc:1397] T 13425835965446758595ff9d8101dd06 P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.004s
I20260620 18:32:07.491552 12131 raft_consensus.cc:348] T 13425835965446758595ff9d8101dd06 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.492028 12131 raft_consensus.cc:374] T 13425835965446758595ff9d8101dd06 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:07.492146 12131 raft_consensus.cc:729] T 13425835965446758595ff9d8101dd06 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:32:07.492733 12131 consensus_queue.cc:260] T 13425835965446758595ff9d8101dd06 P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
@ 0x7fef7ee976db start_thread at ??:0
@ 0x7fef77e2d61f clone at ??:0
I20260620 18:32:07.495582 12131 ts_tablet_manager.cc:1428] T 13425835965446758595ff9d8101dd06 P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260620 18:32:07.495942 12131 tablet_bootstrap.cc:492] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
I20260620 18:32:07.498199 12131 tablet_bootstrap.cc:654] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:07.501919 12128 tablet_bootstrap.cc:492] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:32:07.502238 12128 ts_tablet_manager.cc:1397] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.046s user 0.005s sys 0.006s
I20260620 18:32:07.503557 12128 raft_consensus.cc:348] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.504057 12128 raft_consensus.cc:374] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:07.504607 12128 raft_consensus.cc:729] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:32:07.505333 12122 ts_tablet_manager.cc:1918] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:07.505503 12122 log.cc:1196] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal/wals/360d15e9cf9d4d2aa21dfe7ae0c5c65e
I20260620 18:32:07.505982 12128 consensus_queue.cc:260] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.506315 12127 ts_tablet_manager.cc:1918] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:07.506506 12127 log.cc:1196] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/wal/wals/ac236b968be243ffa51a59f03798498f
I20260620 18:32:07.506965 12127 ts_tablet_manager.cc:1939] T ac236b968be243ffa51a59f03798498f P a0ae894b0758446aa6219aaa5267d1b4: Deleting consensus metadata
I20260620 18:32:07.507038 12128 ts_tablet_manager.cc:1428] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260620 18:32:07.509618 12126 tablet_bootstrap.cc:492] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:32:07.509780 12126 ts_tablet_manager.cc:1397] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.037s user 0.000s sys 0.010s
I20260620 18:32:07.509938 12127 tablet_replica.cc:330] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4: stopping tablet replica
I20260620 18:32:07.510164 12127 raft_consensus.cc:2229] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:32:07.510512 12127 raft_consensus.cc:2258] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:07.510670 12126 raft_consensus.cc:348] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.510927 12126 raft_consensus.cc:374] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:07.511018 12126 raft_consensus.cc:729] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:32:07.511188 12127 ts_tablet_manager.cc:1905] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:07.511545 12122 ts_tablet_manager.cc:1939] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P 9a4ce671d3984e1a8592bc29d45612bc: Deleting consensus metadata
I20260620 18:32:07.518615 12127 ts_tablet_manager.cc:1918] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:07.518771 12127 log.cc:1196] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/wal/wals/360d15e9cf9d4d2aa21dfe7ae0c5c65e
I20260620 18:32:07.519196 12127 ts_tablet_manager.cc:1939] T 360d15e9cf9d4d2aa21dfe7ae0c5c65e P a0ae894b0758446aa6219aaa5267d1b4: Deleting consensus metadata
I20260620 18:32:07.520498 12128 tablet_bootstrap.cc:492] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
I20260620 18:32:07.520677 12126 consensus_queue.cc:260] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.521009 12140 raft_consensus.cc:482] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:07.521205 12140 raft_consensus.cc:504] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.522925 11041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9ec7bf576448abb541eda4936306b2" candidate_uuid: "7696506dcc9746d0aea9b1129a3e9433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc" is_pre_election: true
I20260620 18:32:07.523265 11041 raft_consensus.cc:2454] T 1a9ec7bf576448abb541eda4936306b2 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7696506dcc9746d0aea9b1129a3e9433 in term 0.
I20260620 18:32:07.523313 12140 leader_election.cc:288] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:32:07.523931 10687 leader_election.cc:302] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc; no voters:
I20260620 18:32:07.526086 12140 raft_consensus.cc:2781] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:32:07.526252 12140 raft_consensus.cc:482] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:32:07.526355 12140 raft_consensus.cc:3037] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:07.530373 12140 raft_consensus.cc:504] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.531158 12140 leader_election.cc:288] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 election: Requested vote from peers 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:32:07.523959 12126 ts_tablet_manager.cc:1428] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.014s user 0.000s sys 0.004s
I20260620 18:32:07.532816 12126 tablet_bootstrap.cc:492] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:32:07.532696 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9ec7bf576448abb541eda4936306b2" candidate_uuid: "7696506dcc9746d0aea9b1129a3e9433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4" is_pre_election: true
I20260620 18:32:07.533113 10899 raft_consensus.cc:2454] T 1a9ec7bf576448abb541eda4936306b2 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7696506dcc9746d0aea9b1129a3e9433 in term 0.
I20260620 18:32:07.534955 12126 tablet_bootstrap.cc:654] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:07.536814 11041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9ec7bf576448abb541eda4936306b2" candidate_uuid: "7696506dcc9746d0aea9b1129a3e9433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc"
I20260620 18:32:07.537170 11041 raft_consensus.cc:3037] T 1a9ec7bf576448abb541eda4936306b2 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:07.540354 12126 tablet_bootstrap.cc:492] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:32:07.541091 11041 raft_consensus.cc:2454] T 1a9ec7bf576448abb541eda4936306b2 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7696506dcc9746d0aea9b1129a3e9433 in term 1.
I20260620 18:32:07.541844 12126 ts_tablet_manager.cc:1397] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260620 18:32:07.542762 12126 raft_consensus.cc:348] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.543041 12126 raft_consensus.cc:374] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:07.543118 12126 raft_consensus.cc:729] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:32:07.543391 12126 consensus_queue.cc:260] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.545776 12126 ts_tablet_manager.cc:1428] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260620 18:32:07.546944 12126 tablet_bootstrap.cc:492] T 13425835965446758595ff9d8101dd06 P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:32:07.544158 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9ec7bf576448abb541eda4936306b2" candidate_uuid: "7696506dcc9746d0aea9b1129a3e9433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4"
I20260620 18:32:07.547660 10899 raft_consensus.cc:3037] T 1a9ec7bf576448abb541eda4936306b2 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:07.549017 12126 tablet_bootstrap.cc:654] T 13425835965446758595ff9d8101dd06 P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:07.551337 12126 tablet_bootstrap.cc:492] T 13425835965446758595ff9d8101dd06 P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:32:07.551347 10899 raft_consensus.cc:2454] T 1a9ec7bf576448abb541eda4936306b2 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7696506dcc9746d0aea9b1129a3e9433 in term 1.
I20260620 18:32:07.551482 12126 ts_tablet_manager.cc:1397] T 13425835965446758595ff9d8101dd06 P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260620 18:32:07.552309 12126 raft_consensus.cc:348] T 13425835965446758595ff9d8101dd06 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.552596 12126 raft_consensus.cc:374] T 13425835965446758595ff9d8101dd06 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:07.552685 12126 raft_consensus.cc:729] T 13425835965446758595ff9d8101dd06 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:32:07.553227 12126 consensus_queue.cc:260] T 13425835965446758595ff9d8101dd06 P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.555008 10687 leader_election.cc:302] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:32:07.555397 12141 raft_consensus.cc:2781] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:32:07.555656 12141 raft_consensus.cc:686] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 LEADER]: Becoming Leader. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Running, Role: LEADER
I20260620 18:32:07.556066 12141 consensus_queue.cc:237] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.558670 12126 ts_tablet_manager.cc:1428] T 13425835965446758595ff9d8101dd06 P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260620 18:32:07.560292 12128 tablet_bootstrap.cc:654] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:07.568465 12128 tablet_bootstrap.cc:492] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:32:07.568686 12128 ts_tablet_manager.cc:1397] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.048s user 0.002s sys 0.002s
I20260620 18:32:07.570012 12128 raft_consensus.cc:348] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.570284 12128 raft_consensus.cc:374] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:07.570374 12128 raft_consensus.cc:729] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:32:07.570681 12128 consensus_queue.cc:260] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.572614 12128 ts_tablet_manager.cc:1428] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260620 18:32:07.574649 12131 tablet_bootstrap.cc:492] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:32:07.574807 12131 ts_tablet_manager.cc:1397] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.079s user 0.012s sys 0.023s
I20260620 18:32:07.575623 12131 raft_consensus.cc:348] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.575852 12131 raft_consensus.cc:374] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:07.575958 12131 raft_consensus.cc:729] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:32:07.576242 12131 consensus_queue.cc:260] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.576845 12131 ts_tablet_manager.cc:1428] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:32:07.628486 12121 raft_consensus.cc:482] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:07.628829 12121 raft_consensus.cc:504] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.630152 12121 leader_election.cc:288] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:32:07.631045 10751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13425835965446758595ff9d8101dd06" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:32:07.631404 10751 raft_consensus.cc:2454] T 13425835965446758595ff9d8101dd06 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 0.
I20260620 18:32:07.631363 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13425835965446758595ff9d8101dd06" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4" is_pre_election: true
I20260620 18:32:07.631680 10899 raft_consensus.cc:2454] T 13425835965446758595ff9d8101dd06 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 0.
I20260620 18:32:07.632179 10977 leader_election.cc:302] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc; no voters:
I20260620 18:32:07.632614 12121 raft_consensus.cc:2781] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:32:07.632746 12121 raft_consensus.cc:482] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:32:07.632845 12121 raft_consensus.cc:3037] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:07.635895 12121 raft_consensus.cc:504] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.637635 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13425835965446758595ff9d8101dd06" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4"
I20260620 18:32:07.637949 10899 raft_consensus.cc:3037] T 13425835965446758595ff9d8101dd06 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:07.637807 10751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13425835965446758595ff9d8101dd06" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433"
I20260620 18:32:07.638173 10751 raft_consensus.cc:3037] T 13425835965446758595ff9d8101dd06 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:07.641495 10751 raft_consensus.cc:2454] T 13425835965446758595ff9d8101dd06 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 1.
I20260620 18:32:07.641496 10899 raft_consensus.cc:2454] T 13425835965446758595ff9d8101dd06 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 1.
I20260620 18:32:07.642165 10977 leader_election.cc:302] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc; no voters:
I20260620 18:32:07.642827 12121 leader_election.cc:288] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 election: Requested vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:32:07.643195 12121 raft_consensus.cc:2781] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:32:07.643385 12121 raft_consensus.cc:686] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 LEADER]: Becoming Leader. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Running, Role: LEADER
I20260620 18:32:07.643767 12121 consensus_queue.cc:237] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.705447 11923 raft_consensus.cc:482] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:07.705669 11923 raft_consensus.cc:504] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.706452 11923 leader_election.cc:288] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583), 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961)
I20260620 18:32:07.707511 11041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec6b4c37ff249d3971fe75f54ddeae2" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc" is_pre_election: true
I20260620 18:32:07.707818 11041 raft_consensus.cc:2454] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 0.
I20260620 18:32:07.708809 10751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec6b4c37ff249d3971fe75f54ddeae2" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:32:07.709103 10751 raft_consensus.cc:2454] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 0.
I20260620 18:32:07.709736 10833 leader_election.cc:302] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:32:07.710762 11923 raft_consensus.cc:2781] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:32:07.710891 11923 raft_consensus.cc:482] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:32:07.710976 11923 raft_consensus.cc:3037] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:07.714426 11923 raft_consensus.cc:504] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:07.715891 11041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec6b4c37ff249d3971fe75f54ddeae2" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc"
I20260620 18:32:07.716193 11041 raft_consensus.cc:3037] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:07.719609 11041 raft_consensus.cc:2454] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
I20260620 18:32:07.719977 11923 leader_election.cc:288] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Requested vote from peers 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583), 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961)
I20260620 18:32:07.720783 10751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec6b4c37ff249d3971fe75f54ddeae2" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433"
I20260620 18:32:07.721066 10751 raft_consensus.cc:3037] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:07.723318 10833 leader_election.cc:302] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:32:07.724010 11923 raft_consensus.cc:2781] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:32:07.724298 10751 raft_consensus.cc:2454] T 1ec6b4c37ff249d3971fe75f54ddeae2 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
I20260620 18:32:07.725040 11923 raft_consensus.cc:686] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 LEADER]: Becoming Leader. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Running, Role: LEADER
I20260620 18:32:07.725405 11923 consensus_queue.cc:237] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
W20260620 18:32:07.768906 10976 connection.cc:535] server connection from 127.0.0.1:38108 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:07.769001 10830 connection.cc:535] server connection from 127.0.0.1:53422 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:07.770529 10686 connection.cc:535] client connection to 127.9.231.252:34267 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:07.771430 11099 connection.cc:535] client connection to 127.9.231.252:34267 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:07.772838 11099 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.9.231.252:34267)
W20260620 18:32:07.773550 11099 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.231.252:34267)
W20260620 18:32:07.773765 11099 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.9.231.252:34267)
W20260620 18:32:07.774302 11099 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.9.231.252:34267)
I20260620 18:32:07.899741 12155 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:32:07.900028 12155 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:07.902392 12155 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:07.903807 12023 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:07.906090 12023 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:32:07.906412 12023 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:32:07.906543 12023 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:32:07.907001 12155 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260620 18:32:08.148736 12161 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:08.149035 12161 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:08.149761 12161 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:08.151089 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:32:08.151365 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:32:08.151508 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:32:08.151798 12161 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:32:08.531926 12163 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:32:08.532222 12163 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:08.533079 12163 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:08.535712 12023 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:08.535969 12023 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:08.536139 12023 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:32:08.536518 12163 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260620 18:32:08.575521 12161 consensus_queue.cc:1035] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:32:08.583573 12172 consensus_queue.cc:1035] T 1a9ec7bf576448abb541eda4936306b2 P 7696506dcc9746d0aea9b1129a3e9433 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:32:08.657402 12172 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:08.657719 12172 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:08.658450 12172 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:08.660138 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:32:08.660490 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:32:08.660597 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:32:08.661032 12172 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:32:08.748822 11923 consensus_queue.cc:1035] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:32:08.756223 11922 consensus_queue.cc:1035] T 1ec6b4c37ff249d3971fe75f54ddeae2 P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:32:08.757627 12162 consensus_queue.cc:1035] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:32:08.768949 12162 consensus_queue.cc:1035] T 13425835965446758595ff9d8101dd06 P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:32:09.050365 12185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:32:09.050639 12185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:09.051437 12185 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:09.054303 12023 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:32:09.054653 12023 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:32:09.054792 12023 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:32:09.055212 12185 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260620 18:32:09.708338 12199 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:32:09.708640 12199 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:09.709371 12199 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:09.712394 12023 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:09.712634 12023 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:09.712770 12023 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:32:09.713161 12199 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260620 18:32:10.234576 10143 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
/tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.253:44633
--webserver_interface=127.9.231.253
--webserver_port=37199
--builtin_ntp_servers=127.9.231.212:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260620 18:32:10.686851 12208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:32:10.687481 12208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:32:10.687669 12208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:32:10.698030 12208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:32:10.698227 12208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:32:10.698339 12208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:32:10.698457 12208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:32:10.711737 12208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.231.212:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.253:44633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.231.253
--webserver_port=37199
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 17:42:20 UTC on ccycloud.quasar-rymxqh.root.comops.site
build id 4457681
ASAN enabled
I20260620 18:32:10.713976 12208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:32:10.716586 12208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:32:10.732621 12216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:32:10.732429 12217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:32:10.733827 12219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:32:10.733839 12208 server_base.cc:1034] running on GCE node
I20260620 18:32:10.735108 12208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:32:10.737135 12208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:32:10.738456 12208 hybrid_clock.cc:648] HybridClock initialized: now 1781980330738387 us; error 57 us; skew 500 ppm
I20260620 18:32:10.739020 12208 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 18:32:10.742187 12208 webserver.cc:460] Webserver started at http://127.9.231.253:37199/ using document root <none> and password file <none>
I20260620 18:32:10.743398 12208 fs_manager.cc:359] Metadata directory not provided
I20260620 18:32:10.743548 12208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:32:10.751327 12208 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 18:32:10.755914 12225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:32:10.758123 12208 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260620 18:32:10.758431 12208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data,/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
uuid: "650e0197335644718cd91d5018399ee2"
format_stamp: "Formatted at 2026-06-20 18:31:46 on dist-test-slave-dv4w"
I20260620 18:32:10.759618 12208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:32:10.801209 12208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:32:10.802290 12208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:32:10.802632 12208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:32:10.827837 12208 rpc_server.cc:307] RPC server started. Bound to: 127.9.231.253:44633
I20260620 18:32:10.827862 12276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.231.253:44633 every 8 connection(s)
I20260620 18:32:10.829921 12208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data/info.pb
I20260620 18:32:10.836205 10143 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu as pid 12208
I20260620 18:32:10.839574 12277 sys_catalog.cc:263] Verifying existing consensus state
I20260620 18:32:10.845148 12277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Bootstrap starting.
I20260620 18:32:10.858314 11922 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:32:10.858528 11922 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:10.859369 11922 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:10.860862 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:32:10.861162 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:32:10.861150 12277 log.cc:824] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Log is configured to *not* fsync() on all Append() calls
I20260620 18:32:10.861351 10833 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:32:10.861887 11922 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:32:10.893805 12277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Bootstrap replayed 1/1 log segments. Stats: ops{read=35 overwritten=0 applied=35 ignored=0} inserts{seen=23 ignored=0} mutations{seen=22 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:32:10.894526 12277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Bootstrap complete.
I20260620 18:32:10.904809 12277 raft_consensus.cc:348] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:10.905893 12277 raft_consensus.cc:729] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 650e0197335644718cd91d5018399ee2, State: Initialized, Role: FOLLOWER
I20260620 18:32:10.906687 12277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 35, Last appended: 7.35, Last appended by leader: 35, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:10.907739 12282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:10.908223 12282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:10.909011 12277 sys_catalog.cc:564] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:32:10.924273 12296 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:32:10.924521 12296 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: loading cluster ID for follower catalog manager: success
I20260620 18:32:10.929108 12296 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: acquiring CA information for follower catalog manager: success
I20260620 18:32:10.932726 12296 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:32:10.967516 12208 master_runner.cc:186] Error getting master registration for 127.9.231.252:34267: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111),
I20260620 18:32:10.975772 12208 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:32:11.093277 11089 heartbeater.cc:347] Connected to a master server at 127.9.231.253:44633
I20260620 18:32:11.098975 10946 heartbeater.cc:347] Connected to a master server at 127.9.231.253:44633
I20260620 18:32:11.099086 12242 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" instance_seqno: 1781980309668499) as {username='slave'} at 127.9.231.195:34741; Asking this server to re-register.
I20260620 18:32:11.100375 11089 heartbeater.cc:464] Registering TS with master...
I20260620 18:32:11.102977 12240 ts_manager.cc:194] Registered new tserver with Master: 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:32:11.103086 12241 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" instance_seqno: 1781980309047005) as {username='slave'} at 127.9.231.194:55205; Asking this server to re-register.
I20260620 18:32:11.103964 10946 heartbeater.cc:464] Registering TS with master...
I20260620 18:32:11.105998 12240 ts_manager.cc:194] Registered new tserver with Master: a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:32:11.130165 10798 heartbeater.cc:347] Connected to a master server at 127.9.231.253:44633
I20260620 18:32:11.132587 12240 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" instance_seqno: 1781980308024586) as {username='slave'} at 127.9.231.193:42751; Asking this server to re-register.
I20260620 18:32:11.133543 10798 heartbeater.cc:464] Registering TS with master...
I20260620 18:32:11.135440 12240 ts_manager.cc:194] Registered new tserver with Master: 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961)
I20260620 18:32:11.174165 12305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:32:11.174413 12305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:11.175361 12305 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:11.179718 12023 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:11.186908 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1cd216a992124f0fac2fb65754e6f7df" candidate_term: 11 candidate_status { last_received { term: 10 index: 48 } } ignore_live_leader: false dest_uuid: "650e0197335644718cd91d5018399ee2" is_pre_election: true
I20260620 18:32:11.187664 12252 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cd216a992124f0fac2fb65754e6f7df in term 7.
I20260620 18:32:11.188705 12023 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2; no voters: 4587918520d849eaad95e35037e36608
I20260620 18:32:11.189244 12305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260620 18:32:11.189443 12305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4587918520d849eaad95e35037e36608)
I20260620 18:32:11.189563 12305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 10 FOLLOWER]: Advancing to term 11
I20260620 18:32:11.192760 12305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:11.193686 12305 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 election: Requested vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
I20260620 18:32:11.194689 12252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1cd216a992124f0fac2fb65754e6f7df" candidate_term: 11 candidate_status { last_received { term: 10 index: 48 } } ignore_live_leader: false dest_uuid: "650e0197335644718cd91d5018399ee2"
I20260620 18:32:11.195055 12252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 7 FOLLOWER]: Advancing to term 11
W20260620 18:32:11.196063 12023 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:11.201574 12252 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cd216a992124f0fac2fb65754e6f7df in term 11.
I20260620 18:32:11.202344 12023 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2; no voters: 4587918520d849eaad95e35037e36608
I20260620 18:32:11.202860 12305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 11 FOLLOWER]: Leader election won for term 11
I20260620 18:32:11.203465 12305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 11 LEADER]: Becoming Leader. State: Replica: 1cd216a992124f0fac2fb65754e6f7df, State: Running, Role: LEADER
I20260620 18:32:11.204084 12305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 48, Committed index: 48, Last appended: 10.48, Last appended by leader: 48, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:11.206547 12309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cd216a992124f0fac2fb65754e6f7df. Latest consensus state: current_term: 11 leader_uuid: "1cd216a992124f0fac2fb65754e6f7df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:11.207136 12309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: LEADER
I20260620 18:32:11.208073 12311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:32:11.211638 12311 catalog_manager.cc:669] Loaded metadata for table default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]
I20260620 18:32:11.212201 12311 catalog_manager.cc:669] Loaded metadata for table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]
I20260620 18:32:11.212659 12311 catalog_manager.cc:669] Loaded metadata for table default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74]
I20260620 18:32:11.213136 12311 catalog_manager.cc:669] Loaded metadata for table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]
I20260620 18:32:11.213562 12311 catalog_manager.cc:669] Loaded metadata for table default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3]
I20260620 18:32:11.214037 12311 catalog_manager.cc:669] Loaded metadata for table default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59]
I20260620 18:32:11.214404 12311 catalog_manager.cc:669] Loaded metadata for table default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2]
I20260620 18:32:11.217201 12311 tablet_loader.cc:96] loaded metadata for tablet 13425835965446758595ff9d8101dd06 (table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c])
I20260620 18:32:11.217903 12311 tablet_loader.cc:96] loaded metadata for tablet 1a9ec7bf576448abb541eda4936306b2 (table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770])
I20260620 18:32:11.218429 12311 tablet_loader.cc:96] loaded metadata for tablet 1ec6b4c37ff249d3971fe75f54ddeae2 (table default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047])
I20260620 18:32:11.219663 12311 tablet_loader.cc:96] loaded metadata for tablet 654f8832c4be4d9ea5d23b446696a63f (table default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59])
I20260620 18:32:11.220019 12311 tablet_loader.cc:96] loaded metadata for tablet 6a00bd448f6e4d1fbf99e4c11bb6e970 (table default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3])
I20260620 18:32:11.220534 12311 tablet_loader.cc:96] loaded metadata for tablet 8b82c74e9aa6451c8e7c537f7096aa25 (table default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2])
I20260620 18:32:11.220978 12311 tablet_loader.cc:96] loaded metadata for tablet acc2a8b335d14d20bad1719566a27f3d (table default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74])
I20260620 18:32:11.221935 12311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:32:11.223511 12311 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:32:11.223655 12311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:32:11.225741 12311 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:32:11.227057 12311 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Loaded TSK: 0
I20260620 18:32:11.228616 12311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:32:11.229907 12311 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260620 18:32:11.230923 12311 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260620 18:32:11.267671 12037 master_service.cc:946] ReplaceTablet: received request to replace tablet 13425835965446758595ff9d8101dd06 from {username='slave'} at 127.0.0.1:56292
I20260620 18:32:11.268167 12036 master_service.cc:946] ReplaceTablet: received request to replace tablet 1ec6b4c37ff249d3971fe75f54ddeae2 from {username='slave'} at 127.0.0.1:56292
I20260620 18:32:11.268527 12035 master_service.cc:946] ReplaceTablet: received request to replace tablet 1709fe2d11a24f52bd1812bc37e1714b from {username='slave'} at 127.0.0.1:56292
I20260620 18:32:11.268834 12034 master_service.cc:946] ReplaceTablet: received request to replace tablet 1709fe2d11a24f52bd1812bc37e1714b from {username='slave'} at 127.0.0.1:56292
I20260620 18:32:11.270123 12035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56292:
table { table_name: "default.Table_f27c96a5d11944f6a75123e42b5b2ce5" } modify_external_catalogs: true
I20260620 18:32:11.270306 12033 master_service.cc:946] ReplaceTablet: received request to replace tablet b7a80fba1822404a98459c47faef4680 from {username='slave'} at 127.0.0.1:56292
I20260620 18:32:11.270560 12035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56292:
table { table_name: "default.Table_f27c96a5d11944f6a75123e42b5b2ce5" } modify_external_catalogs: true
I20260620 18:32:11.271452 12033 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56292:
table { table_name: "default.Table_269d7d1648864d4e8b29b944b7115e6a" } new_table_name: "default.Table_45bc016e4a9b4288a42f33020e656b17" modify_external_catalogs: true
I20260620 18:32:11.272718 12032 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56292:
name: "default.Table_1ba08641833a48838fae05c1f72333e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:11.273562 12031 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56292:
table { table_name: "default.Table_23d8d896ffe7406ca154449fd1a1eeab" } new_table_name: "default.Table_042b39ab8bd94475b7844eed01c35807" modify_external_catalogs: true
I20260620 18:32:11.274186 12030 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56292:
name: "default.Table_aa7e29fa2b8e4974a8ece0e46aba222a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:11.275383 12029 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56292:
name: "default.Table_16d5e29e7ed8458180884abae667bf9d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:11.276647 12028 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56292:
name: "default.Table_08795a50aec84f85bc35ae8025674185"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:11.276964 12034 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56292:
table { table_name: "default.Table_009e06deb1f84f74b4a32287a8c44b5a" } new_table_name: "default.Table_7105b83c587d4cf0a6add4272b0defc6" modify_external_catalogs: true
F20260620 18:32:11.286078 12093 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_269d7d1648864d4e8b29b944b7115e6a
*** Check failure stack trace: ***
W20260620 18:32:11.287912 12023 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df -> Peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Couldn't send request to peer 4587918520d849eaad95e35037e36608. Status: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7f866b7e59ad google::LogMessage::Fail() at ??:0
I20260620 18:32:11.290216 12252 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Refusing update from remote peer 1cd216a992124f0fac2fb65754e6f7df: Log matching property violated. Preceding OpId in replica: term: 7 index: 35. Preceding OpId from leader: term: 11 index: 50. (index mismatch)
@ 0x7f866b7e888a google::LogMessage::SendToLog() at ??:0
@ 0x7f866b7e54de google::LogMessage::Flush() at ??:0
I20260620 18:32:11.292685 12309 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [LEADER]: Connected to new peer: Peer: permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 35, Time since last communication: 0.000s
@ 0x7f866b7e56d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f8681383ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f8681382901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f868138966c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f86813894e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f866e40feaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f86716e96db start_thread at ??:0
@ 0x7f866a67f61f clone at ??:0
I20260620 18:32:11.361227 12322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 11 leader_uuid: "1cd216a992124f0fac2fb65754e6f7df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:11.361869 12322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:11.362418 12282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cd216a992124f0fac2fb65754e6f7df. Latest consensus state: current_term: 11 leader_uuid: "1cd216a992124f0fac2fb65754e6f7df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:11.362715 12282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:11.363302 12318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 11 leader_uuid: "1cd216a992124f0fac2fb65754e6f7df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:11.363685 12318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:11.364094 12325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 11 leader_uuid: "1cd216a992124f0fac2fb65754e6f7df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:11.364581 12325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:11.371826 12315 mvcc.cc:205] Tried to move back new op lower bound from 7298991421003726848 to 7298991395951464448. Current Snapshot: MvccSnapshot[applied={T|T < 7298991421003726848 or (T in {7298991421003726848})}]
W20260620 18:32:11.589272 12226 connection.cc:535] server connection from 127.0.0.1:56198 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:11.589740 10831 connection.cc:535] client connection to 127.9.231.254:43203 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:11.589944 11098 connection.cc:535] client connection to 127.9.231.254:43203 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:11.590790 11098 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.9.231.254:43203)
W20260620 18:32:11.591084 10976 connection.cc:535] client connection to 127.9.231.254:43203 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:11.591333 11098 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.231.254:43203)
W20260620 18:32:11.591615 11098 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.9.231.254:43203)
W20260620 18:32:11.592051 11098 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.9.231.254:43203)
I20260620 18:32:11.801222 12325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1cd216a992124f0fac2fb65754e6f7df)
I20260620 18:32:11.801621 12325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:11.804785 12325 leader_election.cc:288] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:11.806767 12227 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
W20260620 18:32:11.809392 12228 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:32:11.809859 12227 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
I20260620 18:32:11.810233 12227 leader_election.cc:302] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 650e0197335644718cd91d5018399ee2; no voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608
I20260620 18:32:11.812369 12325 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260620 18:32:12.377346 12342 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1cd216a992124f0fac2fb65754e6f7df)
I20260620 18:32:12.377768 12342 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:12.378939 12342 leader_election.cc:288] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:12.382103 12228 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:32:12.382759 12227 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
I20260620 18:32:12.382893 12227 leader_election.cc:302] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 650e0197335644718cd91d5018399ee2; no voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608
I20260620 18:32:12.383261 12342 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260620 18:32:12.485469 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 11: this message will repeat every 5th retry.
I20260620 18:32:13.176700 12352 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1cd216a992124f0fac2fb65754e6f7df)
I20260620 18:32:13.176997 12352 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:13.178112 12352 leader_election.cc:288] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:13.181819 12228 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:32:13.182159 12227 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
I20260620 18:32:13.182399 12227 leader_election.cc:302] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 650e0197335644718cd91d5018399ee2; no voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608
I20260620 18:32:13.182924 12352 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260620 18:32:13.458405 10143 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu pid 12208
I20260620 18:32:13.474771 12272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 18:32:13.611471 10143 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu with pid 12208
W20260620 18:32:13.634099 10686 connection.cc:535] client connection to 127.9.231.253:44633 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:13.634172 10977 connection.cc:535] client connection to 127.9.231.253:44633 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:14.230780 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 6: this message will repeat every 5th retry.
W20260620 18:32:14.526999 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 6: this message will repeat every 5th retry.
I20260620 18:32:14.543057 10143 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
/tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.253:44633
--webserver_interface=127.9.231.253
--webserver_port=37199
--builtin_ntp_servers=127.9.231.212:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260620 18:32:14.917881 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 6: this message will repeat every 5th retry.
I20260620 18:32:14.940927 12365 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:14.941128 12365 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:14.941938 12365 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:14.943071 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:32:14.943287 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:32:14.943427 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:32:14.943682 12365 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:32:14.986497 12368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:32:14.986929 12368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:32:14.987059 12368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:32:14.995591 12368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:32:14.995762 12368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:32:14.995812 12368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:32:14.995900 12368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:32:15.007052 12368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.231.212:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.253:44633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.231.253
--webserver_port=37199
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 17:42:20 UTC on ccycloud.quasar-rymxqh.root.comops.site
build id 4457681
ASAN enabled
I20260620 18:32:15.009259 12368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:32:15.012645 12368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:32:15.028644 12374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:32:15.028690 12377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:32:15.028615 12375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:32:15.029698 12368 server_base.cc:1034] running on GCE node
I20260620 18:32:15.030893 12368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:32:15.032944 12368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:32:15.034268 12368 hybrid_clock.cc:648] HybridClock initialized: now 1781980335034154 us; error 90 us; skew 500 ppm
I20260620 18:32:15.034711 12368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 18:32:15.038064 12368 webserver.cc:460] Webserver started at http://127.9.231.253:37199/ using document root <none> and password file <none>
I20260620 18:32:15.039050 12368 fs_manager.cc:359] Metadata directory not provided
I20260620 18:32:15.039228 12368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:32:15.046918 12368 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 18:32:15.051638 12383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:32:15.053957 12368 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260620 18:32:15.054205 12368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data,/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
uuid: "650e0197335644718cd91d5018399ee2"
format_stamp: "Formatted at 2026-06-20 18:31:46 on dist-test-slave-dv4w"
I20260620 18:32:15.055441 12368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:32:15.087451 12368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:32:15.088657 12368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:32:15.088963 12368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:32:15.114471 12368 rpc_server.cc:307] RPC server started. Bound to: 127.9.231.253:44633
I20260620 18:32:15.114513 12434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.231.253:44633 every 8 connection(s)
I20260620 18:32:15.116612 12368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-1/data/info.pb
I20260620 18:32:15.123315 10143 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu as pid 12368
I20260620 18:32:15.125924 12435 sys_catalog.cc:263] Verifying existing consensus state
I20260620 18:32:15.131044 12435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Bootstrap starting.
I20260620 18:32:15.163303 11089 heartbeater.cc:347] Connected to a master server at 127.9.231.253:44633
I20260620 18:32:15.163522 12435 log.cc:824] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Log is configured to *not* fsync() on all Append() calls
I20260620 18:32:15.168473 10946 heartbeater.cc:347] Connected to a master server at 127.9.231.253:44633
I20260620 18:32:15.173183 10798 heartbeater.cc:347] Connected to a master server at 127.9.231.253:44633
I20260620 18:32:15.208848 12435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Bootstrap replayed 1/1 log segments. Stats: ops{read=51 overwritten=0 applied=48 ignored=0} inserts{seen=29 ignored=0} mutations{seen=37 ignored=0} orphaned_commits=0. Pending: 3 replicates
I20260620 18:32:15.209686 12435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Bootstrap complete.
I20260620 18:32:15.219310 12435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Replica starting. Triggering 3 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:15.223989 12435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Becoming Follower/Learner. State: Replica: 650e0197335644718cd91d5018399ee2, State: Initialized, Role: FOLLOWER
I20260620 18:32:15.225070 12435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 48, Last appended: 11.51, Last appended by leader: 51, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:15.226116 12446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 11 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:15.226558 12446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:15.227639 12435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:32:15.252552 12460 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:32:15.252764 12460 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: loading cluster ID for follower catalog manager: success
I20260620 18:32:15.241106 12368 master_runner.cc:186] Error getting master registration for 127.9.231.254:43203: Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111),
I20260620 18:32:15.258195 12368 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:32:15.258701 12460 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: acquiring CA information for follower catalog manager: success
I20260620 18:32:15.262073 12460 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:32:15.394582 12399 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" instance_seqno: 1781980309047005) as {username='slave'} at 127.9.231.194:36283; Asking this server to re-register.
I20260620 18:32:15.395844 10946 heartbeater.cc:464] Registering TS with master...
I20260620 18:32:15.398945 12399 ts_manager.cc:194] Registered new tserver with Master: a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:32:15.670118 12399 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" instance_seqno: 1781980309668499) as {username='slave'} at 127.9.231.195:50577; Asking this server to re-register.
I20260620 18:32:15.670953 11089 heartbeater.cc:464] Registering TS with master...
I20260620 18:32:15.672709 12399 ts_manager.cc:194] Registered new tserver with Master: 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:32:15.677937 12399 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" instance_seqno: 1781980308024586) as {username='slave'} at 127.9.231.193:58457; Asking this server to re-register.
I20260620 18:32:15.678786 10798 heartbeater.cc:464] Registering TS with master...
I20260620 18:32:15.681089 12399 ts_manager.cc:194] Registered new tserver with Master: 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961)
I20260620 18:32:15.705572 12446 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:15.706043 12446 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:15.708587 12446 leader_election.cc:288] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:15.709544 12386 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:32:15.711346 12386 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:32:15.711860 12385 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
I20260620 18:32:15.712036 12385 leader_election.cc:302] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 650e0197335644718cd91d5018399ee2; no voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608
I20260620 18:32:15.712550 12446 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260620 18:32:16.283072 12470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:16.283282 12470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:16.284158 12470 leader_election.cc:288] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:16.287447 12386 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:32:16.287716 12385 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
I20260620 18:32:16.287917 12385 leader_election.cc:302] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 650e0197335644718cd91d5018399ee2; no voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608
I20260620 18:32:16.288313 12470 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260620 18:32:16.913919 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 11: this message will repeat every 5th retry.
I20260620 18:32:17.056471 12479 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:32:17.056767 12479 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:17.057565 12479 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:17.059096 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:32:17.059340 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:32:17.059547 10832 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:32:17.059897 12479 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:32:17.344482 12482 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:17.344753 12482 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:17.345662 12482 leader_election.cc:288] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:17.348582 12385 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203): Network error: Client connection negotiation failed: client connection to 127.9.231.254:43203: connect: Connection refused (error 111)
W20260620 18:32:17.348779 12386 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:17.348927 12386 leader_election.cc:302] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 650e0197335644718cd91d5018399ee2; no voters: 1cd216a992124f0fac2fb65754e6f7df, 4587918520d849eaad95e35037e36608
I20260620 18:32:17.349236 12482 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260620 18:32:18.285622 10143 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
/tmp/dist-test-taskkm04Ww/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.254:43203
--webserver_interface=127.9.231.254
--webserver_port=45071
--builtin_ntp_servers=127.9.231.212:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260620 18:32:18.681121 12494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:32:18.681502 12494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:32:18.681663 12494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:32:18.689990 12494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:32:18.690150 12494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:32:18.690346 12494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:32:18.690459 12494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:32:18.700695 12494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.231.212:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:36231
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.231.254:43203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.231.254
--webserver_port=45071
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 17:42:20 UTC on ccycloud.quasar-rymxqh.root.comops.site
build id 4457681
ASAN enabled
I20260620 18:32:18.702569 12494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:32:18.704691 12494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:32:18.718541 12500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:32:18.718611 12503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:32:18.718762 12501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:32:18.719754 12494 server_base.cc:1034] running on GCE node
I20260620 18:32:18.720983 12494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:32:18.722901 12494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:32:18.724310 12494 hybrid_clock.cc:648] HybridClock initialized: now 1781980338724164 us; error 45 us; skew 500 ppm
I20260620 18:32:18.724795 12494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 18:32:18.732329 12494 webserver.cc:460] Webserver started at http://127.9.231.254:45071/ using document root <none> and password file <none>
I20260620 18:32:18.733299 12494 fs_manager.cc:359] Metadata directory not provided
I20260620 18:32:18.733481 12494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:32:18.740036 12494 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260620 18:32:18.743768 12509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:32:18.745949 12494 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260620 18:32:18.746168 12494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data,/tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
uuid: "1cd216a992124f0fac2fb65754e6f7df"
format_stamp: "Formatted at 2026-06-20 18:31:45 on dist-test-slave-dv4w"
I20260620 18:32:18.747306 12494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:32:18.766537 12494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:32:18.767802 12494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:32:18.768199 12494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:32:18.792562 12494 rpc_server.cc:307] RPC server started. Bound to: 127.9.231.254:43203
I20260620 18:32:18.792496 12560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.231.254:43203 every 8 connection(s)
I20260620 18:32:18.794595 12494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/master-0/data/info.pb
I20260620 18:32:18.802233 12561 sys_catalog.cc:263] Verifying existing consensus state
I20260620 18:32:18.802757 10143 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkm04Ww/build/asan/bin/kudu as pid 12494
I20260620 18:32:18.807268 12561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Bootstrap starting.
I20260620 18:32:18.827600 12561 log.cc:824] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Log is configured to *not* fsync() on all Append() calls
I20260620 18:32:18.837512 10945 heartbeater.cc:347] Connected to a master server at 127.9.231.254:43203
I20260620 18:32:18.876883 12561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Bootstrap replayed 1/1 log segments. Stats: ops{read=51 overwritten=0 applied=48 ignored=0} inserts{seen=29 ignored=0} mutations{seen=37 ignored=0} orphaned_commits=0. Pending: 3 replicates
I20260620 18:32:18.877898 12561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: Bootstrap complete.
I20260620 18:32:18.887082 12561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 11 FOLLOWER]: Replica starting. Triggering 3 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:18.891387 12561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 11 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cd216a992124f0fac2fb65754e6f7df, State: Initialized, Role: FOLLOWER
I20260620 18:32:18.892431 12561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 48, Last appended: 11.51, Last appended by leader: 51, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:18.893460 12571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 11 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:18.893987 12571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:18.895004 12561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:32:18.911599 12585 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:32:18.911849 12585 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: loading cluster ID for follower catalog manager: success
I20260620 18:32:18.917012 12585 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: acquiring CA information for follower catalog manager: success
I20260620 18:32:18.924873 12585 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:32:18.932807 10945 heartbeater.cc:347] Connected to a master server at 127.9.231.254:43203
I20260620 18:32:18.936494 12526 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" instance_seqno: 1781980309047005) as {username='slave'} at 127.9.231.194:33749; Asking this server to re-register.
I20260620 18:32:18.937590 10945 heartbeater.cc:464] Registering TS with master...
I20260620 18:32:18.940003 12526 ts_manager.cc:194] Registered new tserver with Master: a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:32:18.945984 12494 master_runner.cc:186] Error getting master registration for 127.9.231.252:34267: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111),
I20260620 18:32:18.953630 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:32:19.023689 12593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:19.023988 12593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:19.024863 12593 leader_election.cc:288] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:19.032739 12386 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:19.036553 12536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "650e0197335644718cd91d5018399ee2" candidate_term: 12 candidate_status { last_received { term: 11 index: 51 } } ignore_live_leader: false dest_uuid: "1cd216a992124f0fac2fb65754e6f7df" is_pre_election: true
I20260620 18:32:19.037531 12536 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 650e0197335644718cd91d5018399ee2 in term 11.
I20260620 18:32:19.038561 12385 leader_election.cc:302] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2; no voters: 4587918520d849eaad95e35037e36608
I20260620 18:32:19.039052 12593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260620 18:32:19.039263 12593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:32:19.039387 12593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 11 FOLLOWER]: Advancing to term 12
I20260620 18:32:19.049880 12593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:19.053494 12593 leader_election.cc:288] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 election: Requested vote from peers 1cd216a992124f0fac2fb65754e6f7df (127.9.231.254:43203), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
I20260620 18:32:19.053723 12536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "650e0197335644718cd91d5018399ee2" candidate_term: 12 candidate_status { last_received { term: 11 index: 51 } } ignore_live_leader: false dest_uuid: "1cd216a992124f0fac2fb65754e6f7df"
I20260620 18:32:19.054109 12536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 11 FOLLOWER]: Advancing to term 12
W20260620 18:32:19.060213 12386 leader_election.cc:334] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:19.061257 12536 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 650e0197335644718cd91d5018399ee2 in term 12.
I20260620 18:32:19.062083 12385 leader_election.cc:302] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df, 650e0197335644718cd91d5018399ee2; no voters: 4587918520d849eaad95e35037e36608
I20260620 18:32:19.062804 12593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 12 FOLLOWER]: Leader election won for term 12
I20260620 18:32:19.068682 12593 raft_consensus.cc:686] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [term 12 LEADER]: Becoming Leader. State: Replica: 650e0197335644718cd91d5018399ee2, State: Running, Role: LEADER
I20260620 18:32:19.069782 12593 consensus_queue.cc:237] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 48, Committed index: 48, Last appended: 11.51, Last appended by leader: 51, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:19.075362 12599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 650e0197335644718cd91d5018399ee2. Latest consensus state: current_term: 12 leader_uuid: "650e0197335644718cd91d5018399ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:19.075953 12599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: LEADER
I20260620 18:32:19.125774 11088 heartbeater.cc:347] Connected to a master server at 127.9.231.254:43203
I20260620 18:32:19.127835 12526 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" instance_seqno: 1781980309668499) as {username='slave'} at 127.9.231.195:41735; Asking this server to re-register.
I20260620 18:32:19.128700 11088 heartbeater.cc:464] Registering TS with master...
I20260620 18:32:19.130239 12526 ts_manager.cc:194] Registered new tserver with Master: 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:32:19.179522 10797 heartbeater.cc:347] Connected to a master server at 127.9.231.254:43203
I20260620 18:32:19.181610 12526 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" instance_seqno: 1781980308024586) as {username='slave'} at 127.9.231.193:59059; Asking this server to re-register.
I20260620 18:32:19.182461 10797 heartbeater.cc:464] Registering TS with master...
I20260620 18:32:19.184329 12526 ts_manager.cc:194] Registered new tserver with Master: 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961)
I20260620 18:32:19.469308 12604 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:19.469606 12604 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:19.470508 12604 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:19.471948 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:32:19.472242 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:32:19.472395 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:32:19.472757 12604 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:32:19.515570 12386 consensus_peers.cc:487] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 -> Peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Couldn't send request to peer 4587918520d849eaad95e35037e36608. Status: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 18:32:19.528945 12536 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Refusing update from remote peer 650e0197335644718cd91d5018399ee2: Log matching property violated. Preceding OpId in replica: term: 11 index: 51. Preceding OpId from leader: term: 12 index: 52. (index mismatch)
I20260620 18:32:19.530251 12599 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 52, Last known committed idx: 48, Time since last communication: 0.000s
I20260620 18:32:19.535084 12605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 650e0197335644718cd91d5018399ee2. Latest consensus state: current_term: 12 leader_uuid: "650e0197335644718cd91d5018399ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:19.535466 12605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:19.542248 12599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 12 leader_uuid: "650e0197335644718cd91d5018399ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:19.542626 12599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: LEADER
I20260620 18:32:19.542376 12613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 12 leader_uuid: "650e0197335644718cd91d5018399ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:19.542444 12614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 12 leader_uuid: "650e0197335644718cd91d5018399ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:19.542517 12605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 12 leader_uuid: "650e0197335644718cd91d5018399ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } } }
I20260620 18:32:19.542781 12613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2 [sys.catalog]: This master's current role is: LEADER
I20260620 18:32:19.542853 12614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:19.542891 12605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:32:19.601333 12601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:32:19.604068 12601 catalog_manager.cc:669] Loaded metadata for table default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]
I20260620 18:32:19.604593 12601 catalog_manager.cc:669] Loaded metadata for table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770]
I20260620 18:32:19.604871 12601 catalog_manager.cc:669] Loaded metadata for table default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74]
I20260620 18:32:19.605144 12601 catalog_manager.cc:669] Loaded metadata for table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]
I20260620 18:32:19.605500 12601 catalog_manager.cc:669] Loaded metadata for table default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3]
I20260620 18:32:19.605798 12601 catalog_manager.cc:669] Loaded metadata for table default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59]
I20260620 18:32:19.606065 12601 catalog_manager.cc:669] Loaded metadata for table default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2]
I20260620 18:32:19.608078 12601 tablet_loader.cc:96] loaded metadata for tablet 1a9ec7bf576448abb541eda4936306b2 (table default.table_f27c96a5d11944f6a75123e42b5b2ce5 [id=3b76eebe1edc48a0923cb81e36eb2770])
I20260620 18:32:19.609195 12601 tablet_loader.cc:96] loaded metadata for tablet 654f8832c4be4d9ea5d23b446696a63f (table default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59])
I20260620 18:32:19.609428 12601 tablet_loader.cc:96] loaded metadata for tablet 6a00bd448f6e4d1fbf99e4c11bb6e970 (table default.table_269d7d1648864d4e8b29b944b7115e6a [id=963ec1c8d90b48ac9b0124b31d53cfe3])
I20260620 18:32:19.609761 12601 tablet_loader.cc:96] loaded metadata for tablet 8b82c74e9aa6451c8e7c537f7096aa25 (table default.table_23d8d896ffe7406ca154449fd1a1eeab [id=f01b90a2c4f74fb9aacaa125e3b4a9f2])
I20260620 18:32:19.610131 12601 tablet_loader.cc:96] loaded metadata for tablet acc2a8b335d14d20bad1719566a27f3d (table default.table_be97363936284e2faa8400049925a98c [id=4f2995259ed3447491bbea0b9afa3c74])
I20260620 18:32:19.610694 12601 tablet_loader.cc:96] loaded metadata for tablet fa1ab5c4ea564d21ab2ff8928077ad98 (table default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c])
I20260620 18:32:19.610888 12601 tablet_loader.cc:96] loaded metadata for tablet fb42c99f96e744288f62126d289890df (table default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047])
I20260620 18:32:19.611374 12601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:32:19.612673 12601 catalog_manager.cc:1259] Loaded cluster ID: 43c29a2ccaf544f8910abc707edbfad4
I20260620 18:32:19.612747 12601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:32:19.614140 12601 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:32:19.615319 12601 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Loaded TSK: 0
I20260620 18:32:19.616336 12601 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:32:19.617338 12601 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260620 18:32:19.618257 12601 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260620 18:32:19.711216 12399 master_service.cc:946] ReplaceTablet: received request to replace tablet 654f8832c4be4d9ea5d23b446696a63f from {username='slave'} at 127.0.0.1:43840
I20260620 18:32:19.711295 12400 master_service.cc:946] ReplaceTablet: received request to replace tablet b7a80fba1822404a98459c47faef4680 from {username='slave'} at 127.0.0.1:43840
I20260620 18:32:19.711552 12398 master_service.cc:946] ReplaceTablet: received request to replace tablet 1709fe2d11a24f52bd1812bc37e1714b from {username='slave'} at 127.0.0.1:43840
I20260620 18:32:19.712911 12396 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43840:
table { table_name: "default.Table_88afa4cb37564293a220876026ee1931" } modify_external_catalogs: true
I20260620 18:32:19.713428 12396 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43840:
table { table_name: "default.Table_88afa4cb37564293a220876026ee1931" } modify_external_catalogs: true
I20260620 18:32:19.713279 12397 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43840:
name: "default.Table_e1ad292ef5e24a628dcf2d699e127e57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:19.713922 12400 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43840:
name: "default.Table_fce67ec520194c7fabdca1fce51af8a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:19.714453 12398 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43840:
table { table_name: "default.Table_9e9a3aec7a67407ba81fe1f6797b55f6" } new_table_name: "default.Table_70f8c2fd535343bfb5649c75f207510a" modify_external_catalogs: true
I20260620 18:32:19.714362 12395 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43840:
name: "default.Table_e4c5c6935f7a48cebd0f828771b1fe01"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:19.715061 12394 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43840:
name: "default.Table_5854ef7ddc554da59e75ed6ff18718ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:19.717033 12392 master_service.cc:946] ReplaceTablet: received request to replace tablet 1ec6b4c37ff249d3971fe75f54ddeae2 from {username='slave'} at 127.0.0.1:43840
I20260620 18:32:19.717203 12393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43840:
name: "default.Table_1ba08641833a48838fae05c1f72333e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:32:19.717947 12392 master_service.cc:946] ReplaceTablet: received request to replace tablet 13425835965446758595ff9d8101dd06 from {username='slave'} at 127.0.0.1:43840
I20260620 18:32:19.718818 12392 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43840:
table { table_name: "default.Table_23d8d896ffe7406ca154449fd1a1eeab" } new_table_name: "default.Table_042b39ab8bd94475b7844eed01c35807" modify_external_catalogs: true
I20260620 18:32:19.719254 12391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43840:
name: "default.Table_aa7e29fa2b8e4974a8ece0e46aba222a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260620 18:32:19.722673 12459 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_269d7d1648864d4e8b29b944b7115e6a
*** Check failure stack trace: ***
@ 0x7f64088249ad google::LogMessage::Fail() at ??:0
@ 0x7f640882788a google::LogMessage::SendToLog() at ??:0
@ 0x7f64088244de google::LogMessage::Flush() at ??:0
I20260620 18:32:19.731972 12399 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 650e0197335644718cd91d5018399ee2: Sending DeleteTablet for 3 replicas of tablet 654f8832c4be4d9ea5d23b446696a63f
@ 0x7f64088246d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f641e3c2ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f641e3c1901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f641e3c866c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260620 18:32:19.755789 10731 tablet_service.cc:1514] Processing DeleteTablet for tablet 654f8832c4be4d9ea5d23b446696a63f with delete_type TABLET_DATA_DELETED (replaced by tablet 6270d8c0017d4f82860f578d98b9071b) from {username='slave'} at 127.0.0.1:37976
I20260620 18:32:19.761549 12399 catalog_manager.cc:6565] ReplaceTablet: tablet 654f8832c4be4d9ea5d23b446696a63f deleted and replaced by tablet 6270d8c0017d4f82860f578d98b9071b
I20260620 18:32:19.762439 12399 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43840:
name: "default.Table_16d5e29e7ed8458180884abae667bf9d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f641e3c84e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260620 18:32:19.774235 10730 tablet_service.cc:1467] Processing CreateTablet for tablet fb42c99f96e744288f62126d289890df (DEFAULT_TABLE table=default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:19.774533 10728 tablet_service.cc:1467] Processing CreateTablet for tablet 6270d8c0017d4f82860f578d98b9071b (DEFAULT_TABLE table=default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:19.774663 10729 tablet_service.cc:1467] Processing CreateTablet for tablet fa1ab5c4ea564d21ab2ff8928077ad98 (DEFAULT_TABLE table=default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:19.774837 10730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb42c99f96e744288f62126d289890df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:19.775096 10728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6270d8c0017d4f82860f578d98b9071b. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260620 18:32:19.780576 10729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa1ab5c4ea564d21ab2ff8928077ad98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:19.781749 12626 tablet_replica.cc:330] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433: stopping tablet replica
I20260620 18:32:19.782042 12626 raft_consensus.cc:2229] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:32:19.782377 12626 raft_consensus.cc:2258] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:19.783140 12626 ts_tablet_manager.cc:1905] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:19.783008 10878 tablet_service.cc:1514] Processing DeleteTablet for tablet 654f8832c4be4d9ea5d23b446696a63f with delete_type TABLET_DATA_DELETED (replaced by tablet 6270d8c0017d4f82860f578d98b9071b) from {username='slave'} at 127.0.0.1:51288
I20260620 18:32:19.783890 10879 tablet_service.cc:1467] Processing CreateTablet for tablet fa1ab5c4ea564d21ab2ff8928077ad98 (DEFAULT_TABLE table=default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:19.783885 10876 tablet_service.cc:1467] Processing CreateTablet for tablet fb42c99f96e744288f62126d289890df (DEFAULT_TABLE table=default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:19.784431 10876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb42c99f96e744288f62126d289890df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:19.784648 10879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa1ab5c4ea564d21ab2ff8928077ad98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:19.789846 12626 ts_tablet_manager.cc:1918] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:19.790017 12626 log.cc:1196] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-0/wal/wals/654f8832c4be4d9ea5d23b446696a63f
I20260620 18:32:19.790586 12626 ts_tablet_manager.cc:1939] T 654f8832c4be4d9ea5d23b446696a63f P 7696506dcc9746d0aea9b1129a3e9433: Deleting consensus metadata
I20260620 18:32:19.791003 10877 tablet_service.cc:1467] Processing CreateTablet for tablet 6270d8c0017d4f82860f578d98b9071b (DEFAULT_TABLE table=default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:19.791570 10877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6270d8c0017d4f82860f578d98b9071b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:19.792222 12386 catalog_manager.cc:4841] TS 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): tablet 654f8832c4be4d9ea5d23b446696a63f (table default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59]) successfully deleted
I20260620 18:32:19.792438 12631 tablet_bootstrap.cc:492] T fa1ab5c4ea564d21ab2ff8928077ad98 P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:32:19.794358 12631 tablet_bootstrap.cc:654] T fa1ab5c4ea564d21ab2ff8928077ad98 P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:19.799369 12631 tablet_bootstrap.cc:492] T fa1ab5c4ea564d21ab2ff8928077ad98 P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:32:19.799620 12631 ts_tablet_manager.cc:1397] T fa1ab5c4ea564d21ab2ff8928077ad98 P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.007s
I20260620 18:32:19.800837 12631 raft_consensus.cc:348] T fa1ab5c4ea564d21ab2ff8928077ad98 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.801224 12631 raft_consensus.cc:374] T fa1ab5c4ea564d21ab2ff8928077ad98 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:19.801337 12631 raft_consensus.cc:729] T fa1ab5c4ea564d21ab2ff8928077ad98 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:32:19.801743 12631 consensus_queue.cc:260] T fa1ab5c4ea564d21ab2ff8928077ad98 P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.802480 10798 heartbeater.cc:503] Master 127.9.231.253:44633 was elected leader, sending a full tablet report...
I20260620 18:32:19.804409 12631 ts_tablet_manager.cc:1428] T fa1ab5c4ea564d21ab2ff8928077ad98 P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
@ 0x7f640b44eeaf kudu::Thread::SuperviseThread() at ??:0
I20260620 18:32:19.804807 12631 tablet_bootstrap.cc:492] T fb42c99f96e744288f62126d289890df P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:32:19.806905 12631 tablet_bootstrap.cc:654] T fb42c99f96e744288f62126d289890df P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
@ 0x7f640e7286db start_thread at ??:0
I20260620 18:32:19.808053 11022 tablet_service.cc:1514] Processing DeleteTablet for tablet 654f8832c4be4d9ea5d23b446696a63f with delete_type TABLET_DATA_DELETED (replaced by tablet 6270d8c0017d4f82860f578d98b9071b) from {username='slave'} at 127.0.0.1:36540
I20260620 18:32:19.808470 11021 tablet_service.cc:1467] Processing CreateTablet for tablet fa1ab5c4ea564d21ab2ff8928077ad98 (DEFAULT_TABLE table=default.table_009e06deb1f84f74b4a32287a8c44b5a [id=83dd7dd7bed24f758c98e9019c63ef7c]), partition=RANGE (key) PARTITION UNBOUNDED
@ 0x7f64076be61f clone at ??:0
I20260620 18:32:19.809090 11021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa1ab5c4ea564d21ab2ff8928077ad98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:19.811318 12632 tablet_bootstrap.cc:492] T fb42c99f96e744288f62126d289890df P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
I20260620 18:32:19.812739 12630 tablet_replica.cc:330] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4: stopping tablet replica
I20260620 18:32:19.812721 11019 tablet_service.cc:1467] Processing CreateTablet for tablet 6270d8c0017d4f82860f578d98b9071b (DEFAULT_TABLE table=default.table_88afa4cb37564293a220876026ee1931 [id=df30a4bddc9b4511989d6c37feb16d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:19.812968 12630 raft_consensus.cc:2229] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:32:19.813252 12630 raft_consensus.cc:2258] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:19.813279 11019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6270d8c0017d4f82860f578d98b9071b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:19.813985 12630 ts_tablet_manager.cc:1905] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:19.816151 12632 tablet_bootstrap.cc:654] T fb42c99f96e744288f62126d289890df P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:19.808470 11018 tablet_service.cc:1467] Processing CreateTablet for tablet fb42c99f96e744288f62126d289890df (DEFAULT_TABLE table=default.table_9e9a3aec7a67407ba81fe1f6797b55f6 [id=2a7c8da0fac442988ce595b8b74d4047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:32:19.817126 11018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb42c99f96e744288f62126d289890df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:32:19.821022 12635 tablet_replica.cc:330] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc: stopping tablet replica
I20260620 18:32:19.821319 12635 raft_consensus.cc:2229] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:32:19.821835 12635 raft_consensus.cc:2258] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:32:19.822487 12635 ts_tablet_manager.cc:1905] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:32:19.823932 12630 ts_tablet_manager.cc:1918] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:19.824083 12630 log.cc:1196] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-1/wal/wals/654f8832c4be4d9ea5d23b446696a63f
I20260620 18:32:19.827512 12630 ts_tablet_manager.cc:1939] T 654f8832c4be4d9ea5d23b446696a63f P a0ae894b0758446aa6219aaa5267d1b4: Deleting consensus metadata
I20260620 18:32:19.831825 12637 tablet_bootstrap.cc:492] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
I20260620 18:32:19.833709 12637 tablet_bootstrap.cc:654] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:19.836285 12632 tablet_bootstrap.cc:492] T fb42c99f96e744288f62126d289890df P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:32:19.836506 12632 ts_tablet_manager.cc:1397] T fb42c99f96e744288f62126d289890df P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.000s
I20260620 18:32:19.837577 12632 raft_consensus.cc:348] T fb42c99f96e744288f62126d289890df P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.837859 12632 raft_consensus.cc:374] T fb42c99f96e744288f62126d289890df P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:19.837954 12632 raft_consensus.cc:729] T fb42c99f96e744288f62126d289890df P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:32:19.838286 12632 consensus_queue.cc:260] T fb42c99f96e744288f62126d289890df P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.839653 12637 tablet_bootstrap.cc:492] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:32:19.839870 12637 ts_tablet_manager.cc:1397] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260620 18:32:19.841840 12637 raft_consensus.cc:348] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.842123 12637 raft_consensus.cc:374] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:19.842221 12637 raft_consensus.cc:729] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:32:19.842573 12635 ts_tablet_manager.cc:1918] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260620 18:32:19.842697 12635 log.cc:1196] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc: Deleting WAL directory at /tmp/dist-test-taskkm04Ww/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1781980288659171-10143-0/minicluster-data/ts-2/wal/wals/654f8832c4be4d9ea5d23b446696a63f
I20260620 18:32:19.843184 12635 ts_tablet_manager.cc:1939] T 654f8832c4be4d9ea5d23b446696a63f P 9a4ce671d3984e1a8592bc29d45612bc: Deleting consensus metadata
I20260620 18:32:19.843570 12632 ts_tablet_manager.cc:1428] T fb42c99f96e744288f62126d289890df P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260620 18:32:19.844357 12631 tablet_bootstrap.cc:492] T fb42c99f96e744288f62126d289890df P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:32:19.844542 12631 ts_tablet_manager.cc:1397] T fb42c99f96e744288f62126d289890df P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.040s user 0.005s sys 0.005s
I20260620 18:32:19.845439 12631 raft_consensus.cc:348] T fb42c99f96e744288f62126d289890df P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.845708 12631 raft_consensus.cc:374] T fb42c99f96e744288f62126d289890df P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:19.845775 12631 raft_consensus.cc:729] T fb42c99f96e744288f62126d289890df P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:32:19.846078 12631 consensus_queue.cc:260] T fb42c99f96e744288f62126d289890df P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.846645 12631 ts_tablet_manager.cc:1428] T fb42c99f96e744288f62126d289890df P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:32:19.848748 12631 tablet_bootstrap.cc:492] T 6270d8c0017d4f82860f578d98b9071b P 7696506dcc9746d0aea9b1129a3e9433: Bootstrap starting.
I20260620 18:32:19.850420 12631 tablet_bootstrap.cc:654] T 6270d8c0017d4f82860f578d98b9071b P 7696506dcc9746d0aea9b1129a3e9433: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:19.851444 12637 consensus_queue.cc:260] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.852222 12632 tablet_bootstrap.cc:492] T 6270d8c0017d4f82860f578d98b9071b P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
I20260620 18:32:19.854583 12632 tablet_bootstrap.cc:654] T 6270d8c0017d4f82860f578d98b9071b P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:19.855536 11089 heartbeater.cc:503] Master 127.9.231.253:44633 was elected leader, sending a full tablet report...
I20260620 18:32:19.855669 12637 ts_tablet_manager.cc:1428] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.016s user 0.002s sys 0.002s
I20260620 18:32:19.856036 12637 tablet_bootstrap.cc:492] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
I20260620 18:32:19.858323 12637 tablet_bootstrap.cc:654] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:19.861084 12637 tablet_bootstrap.cc:492] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:32:19.861284 12637 ts_tablet_manager.cc:1397] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260620 18:32:19.862277 12637 raft_consensus.cc:348] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.862610 12637 raft_consensus.cc:374] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:19.862695 12637 raft_consensus.cc:729] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:32:19.863255 12637 consensus_queue.cc:260] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.867233 12637 ts_tablet_manager.cc:1428] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260620 18:32:19.867671 12637 tablet_bootstrap.cc:492] T fa1ab5c4ea564d21ab2ff8928077ad98 P 9a4ce671d3984e1a8592bc29d45612bc: Bootstrap starting.
I20260620 18:32:19.869957 12637 tablet_bootstrap.cc:654] T fa1ab5c4ea564d21ab2ff8928077ad98 P 9a4ce671d3984e1a8592bc29d45612bc: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:19.871037 12632 tablet_bootstrap.cc:492] T 6270d8c0017d4f82860f578d98b9071b P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:32:19.878690 12631 tablet_bootstrap.cc:492] T 6270d8c0017d4f82860f578d98b9071b P 7696506dcc9746d0aea9b1129a3e9433: No bootstrap required, opened a new log
I20260620 18:32:19.878917 12631 ts_tablet_manager.cc:1397] T 6270d8c0017d4f82860f578d98b9071b P 7696506dcc9746d0aea9b1129a3e9433: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.012s
I20260620 18:32:19.879869 12631 raft_consensus.cc:348] T 6270d8c0017d4f82860f578d98b9071b P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.880138 12631 raft_consensus.cc:374] T 6270d8c0017d4f82860f578d98b9071b P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:19.881393 12632 ts_tablet_manager.cc:1397] T 6270d8c0017d4f82860f578d98b9071b P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.004s
I20260620 18:32:19.883103 12632 raft_consensus.cc:348] T 6270d8c0017d4f82860f578d98b9071b P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.887641 12632 raft_consensus.cc:374] T 6270d8c0017d4f82860f578d98b9071b P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:19.887816 12632 raft_consensus.cc:729] T 6270d8c0017d4f82860f578d98b9071b P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:32:19.880224 12631 raft_consensus.cc:729] T 6270d8c0017d4f82860f578d98b9071b P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7696506dcc9746d0aea9b1129a3e9433, State: Initialized, Role: FOLLOWER
I20260620 18:32:19.888298 12632 consensus_queue.cc:260] T 6270d8c0017d4f82860f578d98b9071b P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.889066 12631 consensus_queue.cc:260] T 6270d8c0017d4f82860f578d98b9071b P 7696506dcc9746d0aea9b1129a3e9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.894515 12631 ts_tablet_manager.cc:1428] T 6270d8c0017d4f82860f578d98b9071b P 7696506dcc9746d0aea9b1129a3e9433: Time spent starting tablet: real 0.015s user 0.003s sys 0.000s
I20260620 18:32:19.907622 12641 raft_consensus.cc:482] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:19.907979 12641 raft_consensus.cc:504] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.909322 12641 leader_election.cc:288] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:32:19.910441 10751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb42c99f96e744288f62126d289890df" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:32:19.910794 10751 raft_consensus.cc:2454] T fb42c99f96e744288f62126d289890df P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 0.
I20260620 18:32:19.911867 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb42c99f96e744288f62126d289890df" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4" is_pre_election: true
I20260620 18:32:19.912690 10977 leader_election.cc:302] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc; no voters:
I20260620 18:32:19.912201 10899 raft_consensus.cc:2454] T fb42c99f96e744288f62126d289890df P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 0.
I20260620 18:32:19.913838 12641 raft_consensus.cc:2781] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:32:19.913995 12641 raft_consensus.cc:482] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:32:19.914112 12641 raft_consensus.cc:3037] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:19.919469 12641 raft_consensus.cc:504] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.936306 12641 leader_election.cc:288] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 election: Requested vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:32:19.937942 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb42c99f96e744288f62126d289890df" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4"
I20260620 18:32:19.938239 12637 tablet_bootstrap.cc:492] T fa1ab5c4ea564d21ab2ff8928077ad98 P 9a4ce671d3984e1a8592bc29d45612bc: No bootstrap required, opened a new log
I20260620 18:32:19.938325 10899 raft_consensus.cc:3037] T fb42c99f96e744288f62126d289890df P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:19.937963 12632 ts_tablet_manager.cc:1428] T 6270d8c0017d4f82860f578d98b9071b P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.056s user 0.006s sys 0.005s
I20260620 18:32:19.938599 12637 ts_tablet_manager.cc:1397] T fa1ab5c4ea564d21ab2ff8928077ad98 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent bootstrapping tablet: real 0.071s user 0.026s sys 0.039s
I20260620 18:32:19.939694 12637 raft_consensus.cc:348] T fa1ab5c4ea564d21ab2ff8928077ad98 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.940007 12637 raft_consensus.cc:374] T fa1ab5c4ea564d21ab2ff8928077ad98 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:19.940405 12632 tablet_bootstrap.cc:492] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4: Bootstrap starting.
I20260620 18:32:19.941709 10899 raft_consensus.cc:2454] T fb42c99f96e744288f62126d289890df P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 1.
I20260620 18:32:19.942286 10978 leader_election.cc:302] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:32:19.942538 12632 tablet_bootstrap.cc:654] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4: Neither blocks nor log segments found. Creating new log.
I20260620 18:32:19.943006 12637 raft_consensus.cc:729] T fa1ab5c4ea564d21ab2ff8928077ad98 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Initialized, Role: FOLLOWER
I20260620 18:32:19.943398 12641 raft_consensus.cc:482] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:19.943380 12637 consensus_queue.cc:260] T fa1ab5c4ea564d21ab2ff8928077ad98 P 9a4ce671d3984e1a8592bc29d45612bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.943660 12641 raft_consensus.cc:504] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.944607 12652 raft_consensus.cc:2781] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:32:19.944772 12637 ts_tablet_manager.cc:1428] T fa1ab5c4ea564d21ab2ff8928077ad98 P 9a4ce671d3984e1a8592bc29d45612bc: Time spent starting tablet: real 0.006s user 0.001s sys 0.002s
I20260620 18:32:19.947351 10751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6270d8c0017d4f82860f578d98b9071b" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:32:19.947821 10751 raft_consensus.cc:2454] T 6270d8c0017d4f82860f578d98b9071b P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 0.
I20260620 18:32:19.948531 10977 leader_election.cc:302] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc; no voters:
I20260620 18:32:19.947392 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb42c99f96e744288f62126d289890df" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433"
I20260620 18:32:19.952039 10750 raft_consensus.cc:3037] T fb42c99f96e744288f62126d289890df P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:19.958630 12653 raft_consensus.cc:2781] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:32:19.958847 12653 raft_consensus.cc:482] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:32:19.959059 12653 raft_consensus.cc:3037] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:19.960418 12652 raft_consensus.cc:686] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [term 1 LEADER]: Becoming Leader. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Running, Role: LEADER
I20260620 18:32:19.961072 12652 consensus_queue.cc:237] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.961738 12641 leader_election.cc:288] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
I20260620 18:32:19.963407 12653 raft_consensus.cc:504] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.956141 10750 raft_consensus.cc:2454] T fb42c99f96e744288f62126d289890df P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 1.
I20260620 18:32:19.973888 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6270d8c0017d4f82860f578d98b9071b" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4" is_pre_election: true
I20260620 18:32:19.974422 10899 raft_consensus.cc:2454] T 6270d8c0017d4f82860f578d98b9071b P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 0.
I20260620 18:32:19.982591 10751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6270d8c0017d4f82860f578d98b9071b" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433"
I20260620 18:32:19.982729 10899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6270d8c0017d4f82860f578d98b9071b" candidate_uuid: "9a4ce671d3984e1a8592bc29d45612bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ae894b0758446aa6219aaa5267d1b4"
I20260620 18:32:19.982887 10751 raft_consensus.cc:3037] T 6270d8c0017d4f82860f578d98b9071b P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:19.982998 10899 raft_consensus.cc:3037] T 6270d8c0017d4f82860f578d98b9071b P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:19.986133 10899 raft_consensus.cc:2454] T 6270d8c0017d4f82860f578d98b9071b P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 1.
I20260620 18:32:19.986148 10751 raft_consensus.cc:2454] T 6270d8c0017d4f82860f578d98b9071b P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a4ce671d3984e1a8592bc29d45612bc in term 1.
I20260620 18:32:19.986822 10978 leader_election.cc:302] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:32:19.987366 12652 raft_consensus.cc:2781] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:32:19.987596 12652 raft_consensus.cc:686] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [term 1 LEADER]: Becoming Leader. State: Replica: 9a4ce671d3984e1a8592bc29d45612bc, State: Running, Role: LEADER
I20260620 18:32:19.987876 12652 consensus_queue.cc:237] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.988317 12632 tablet_bootstrap.cc:492] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4: No bootstrap required, opened a new log
I20260620 18:32:19.988507 12632 ts_tablet_manager.cc:1397] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4: Time spent bootstrapping tablet: real 0.048s user 0.004s sys 0.001s
I20260620 18:32:19.989274 12632 raft_consensus.cc:348] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.989547 12632 raft_consensus.cc:374] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:32:19.989634 12632 raft_consensus.cc:729] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Initialized, Role: FOLLOWER
I20260620 18:32:19.989873 12632 consensus_queue.cc:260] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:19.990550 12632 ts_tablet_manager.cc:1428] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260620 18:32:19.990787 12653 leader_election.cc:288] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [CANDIDATE]: Term 1 election: Requested vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409)
W20260620 18:32:20.104753 10831 connection.cc:535] server connection from 127.0.0.1:51288 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:20.104811 10978 connection.cc:535] server connection from 127.0.0.1:36540 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:20.105314 10832 connection.cc:535] client connection to 127.9.231.253:44633 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:20.105943 10977 connection.cc:535] client connection to 127.9.231.253:44633 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:20.106839 11099 connection.cc:535] client connection to 127.9.231.253:44633 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 18:32:20.107666 11099 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.9.231.253:44633)
W20260620 18:32:20.108424 11099 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.9.231.253:44633)
W20260620 18:32:20.108671 11099 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.231.253:44633)
W20260620 18:32:20.109206 11099 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.9.231.253:44633)
I20260620 18:32:20.271612 12661 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:32:20.271948 12661 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:20.274194 12661 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:20.275861 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:20.277621 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:20.277966 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:20.278124 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:32:20.278606 12661 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:32:20.601761 12647 raft_consensus.cc:482] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:20.602074 12647 raft_consensus.cc:504] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:20.602979 12647 leader_election.cc:288] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583), 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961)
I20260620 18:32:20.603766 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa1ab5c4ea564d21ab2ff8928077ad98" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433" is_pre_election: true
I20260620 18:32:20.603787 11041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa1ab5c4ea564d21ab2ff8928077ad98" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc" is_pre_election: true
I20260620 18:32:20.604013 10750 raft_consensus.cc:2454] T fa1ab5c4ea564d21ab2ff8928077ad98 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 0.
I20260620 18:32:20.604202 11041 raft_consensus.cc:2454] T fa1ab5c4ea564d21ab2ff8928077ad98 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 0.
I20260620 18:32:20.604635 10832 leader_election.cc:302] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:32:20.605036 12647 raft_consensus.cc:2781] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:32:20.605181 12647 raft_consensus.cc:482] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:32:20.605307 12647 raft_consensus.cc:3037] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:20.607748 12647 raft_consensus.cc:504] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:20.608534 12647 leader_election.cc:288] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Requested vote from peers 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583), 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961)
I20260620 18:32:20.609241 11041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa1ab5c4ea564d21ab2ff8928077ad98" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ce671d3984e1a8592bc29d45612bc"
I20260620 18:32:20.609522 11041 raft_consensus.cc:3037] T fa1ab5c4ea564d21ab2ff8928077ad98 P 9a4ce671d3984e1a8592bc29d45612bc [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:20.609580 10750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa1ab5c4ea564d21ab2ff8928077ad98" candidate_uuid: "a0ae894b0758446aa6219aaa5267d1b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7696506dcc9746d0aea9b1129a3e9433"
I20260620 18:32:20.609887 10750 raft_consensus.cc:3037] T fa1ab5c4ea564d21ab2ff8928077ad98 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:32:20.612895 11041 raft_consensus.cc:2454] T fa1ab5c4ea564d21ab2ff8928077ad98 P 9a4ce671d3984e1a8592bc29d45612bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
I20260620 18:32:20.612923 10750 raft_consensus.cc:2454] T fa1ab5c4ea564d21ab2ff8928077ad98 P 7696506dcc9746d0aea9b1129a3e9433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ae894b0758446aa6219aaa5267d1b4 in term 1.
I20260620 18:32:20.613510 10832 leader_election.cc:302] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433, a0ae894b0758446aa6219aaa5267d1b4; no voters:
I20260620 18:32:20.613881 12647 raft_consensus.cc:2781] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:32:20.614073 12647 raft_consensus.cc:686] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 LEADER]: Becoming Leader. State: Replica: a0ae894b0758446aa6219aaa5267d1b4, State: Running, Role: LEADER
I20260620 18:32:20.614446 12647 consensus_queue.cc:237] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } }
I20260620 18:32:20.755738 12667 consensus_queue.cc:1035] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:32:20.763865 12669 consensus_queue.cc:1035] T 6270d8c0017d4f82860f578d98b9071b P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:32:20.854435 12669 consensus_queue.cc:1035] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:32:20.861637 12669 consensus_queue.cc:1035] T fb42c99f96e744288f62126d289890df P 9a4ce671d3984e1a8592bc29d45612bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260620 18:32:20.921773 12680 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:32:20.921963 12680 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:20.922582 12680 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:20.925843 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:20.926141 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:20.926292 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:32:20.926638 12680 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:32:21.254136 10143 master-stress-test.cc:578] Killing test thread 1/25
I20260620 18:32:21.450114 12684 consensus_queue.cc:1035] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:32:21.456193 12648 consensus_queue.cc:1035] T fa1ab5c4ea564d21ab2ff8928077ad98 P a0ae894b0758446aa6219aaa5267d1b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:32:21.525312 12695 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:32:21.525552 12695 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:21.526675 12695 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:21.530778 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:21.531076 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:21.531193 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:32:21.531497 12695 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:32:21.888893 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 16: this message will repeat every 5th retry.
I20260620 18:32:22.253919 12709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:32:22.254221 12709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:22.255020 12709 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:22.258240 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:22.258508 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:22.258844 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:32:22.259377 12709 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:32:23.241354 12727 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:32:23.241613 12727 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:23.242357 12727 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:23.245455 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:23.245805 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:23.245931 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:32:23.246363 12727 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:32:23.727031 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 11: this message will repeat every 5th retry.
I20260620 18:32:23.959038 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:32:23.959372 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:32:24.054339 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 11: this message will repeat every 5th retry.
I20260620 18:32:24.294078 12720 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:32:24.294456 12720 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:24.296084 12720 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:24.297354 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:32:24.297623 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:32:24.297850 10833 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:32:24.298230 12720 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260620 18:32:25.009841 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 11: this message will repeat every 5th retry.
I20260620 18:32:25.622606 12758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:32:25.622861 12758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:25.623579 12758 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:25.625067 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260620 18:32:25.626628 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:25.626869 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:25.626978 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:32:25.627311 12758 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:32:26.156662 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 16: this message will repeat every 5th retry.
I20260620 18:32:26.685190 12777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:32:26.685637 12777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:26.686566 12777 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:26.689783 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:26.690042 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:26.690193 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:32:26.690500 12777 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:32:28.963588 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:32:28.963901 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:32:29.261082 12810 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:29.261368 12810 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:29.262238 12810 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:29.263639 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:32:29.263892 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:32:29.264022 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:32:29.264467 12810 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:32:30.981792 12833 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:32:30.982048 12833 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:30.982714 12833 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:30.984117 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:32:30.984341 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:32:30.984478 10832 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:32:30.984879 12833 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:32:32.131403 12859 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:32.131681 12859 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:32.132457 12859 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:32.133972 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:32:32.134267 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:32:32.134361 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:32:32.134727 12859 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:32:32.318141 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 21: this message will repeat every 5th retry.
I20260620 18:32:33.968228 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:32:33.968550 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:32:34.152673 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 16: this message will repeat every 5th retry.
W20260620 18:32:34.572798 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 16: this message will repeat every 5th retry.
I20260620 18:32:35.073388 12902 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:32:35.073642 12902 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:35.074395 12902 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:35.076006 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260620 18:32:35.078061 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:32:35.078297 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:32:35.078423 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:32:35.078765 12902 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:32:35.693657 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 16: this message will repeat every 5th retry.
W20260620 18:32:36.061457 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 21: this message will repeat every 5th retry.
I20260620 18:32:37.341859 12936 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:37.342099 12936 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:37.343030 12936 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:37.344658 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:32:37.344907 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:32:37.345052 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:32:37.345397 12936 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:32:38.972458 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:32:38.972733 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:32:41.777928 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 26: this message will repeat every 5th retry.
I20260620 18:32:43.316493 13007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:32:43.316722 13007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:43.317473 13007 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:43.319010 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260620 18:32:43.320861 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:43.321079 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:43.321231 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:32:43.321502 13007 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:32:43.976758 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:32:43.977015 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:32:44.264436 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 21: this message will repeat every 5th retry.
W20260620 18:32:44.553486 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 21: this message will repeat every 5th retry.
I20260620 18:32:44.685525 13023 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:44.685755 13023 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:44.686585 13023 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:44.687671 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:32:44.687929 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:32:44.688046 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:32:44.688395 13023 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:32:46.155413 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 21: this message will repeat every 5th retry.
W20260620 18:32:46.159338 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 26: this message will repeat every 5th retry.
I20260620 18:32:46.753866 12943 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:32:46.754110 12943 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:46.755053 12943 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:46.756014 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:32:46.756403 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:32:46.756541 10832 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:32:46.756784 12943 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:32:48.981820 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:32:48.982232 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:32:50.274520 10944 heartbeater.cc:644] Failed to heartbeat to 127.9.231.252:34267 (251 consecutive failures): Network error: Failed to ping master at 127.9.231.252:34267: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:50.575045 13085 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:50.575287 13085 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:50.576496 13085 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:50.577982 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:32:50.578248 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:32:50.578398 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:32:50.578789 13085 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:32:50.586263 10798 heartbeater.cc:644] Failed to heartbeat to 127.9.231.253:44633 (67 consecutive failures): Network error: Failed to ping master at 127.9.231.253:44633: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:50.726727 11089 heartbeater.cc:644] Failed to heartbeat to 127.9.231.253:44633 (69 consecutive failures): Network error: Failed to ping master at 127.9.231.253:44633: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:32:50.746460 13096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:32:50.746727 13096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:32:50.747510 13096 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:32:50.748921 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260620 18:32:50.750676 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:32:50.750929 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:32:50.751070 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:32:50.751370 13096 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:32:51.816545 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 31: this message will repeat every 5th retry.
W20260620 18:32:53.409328 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 26: this message will repeat every 5th retry.
I20260620 18:32:53.986593 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:32:53.986840 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:32:54.176342 13147 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:32:54.176577 13147 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:54.177371 13147 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:54.178773 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:32:54.178972 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:32:54.179064 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:32:54.179507 13147 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:32:54.439020 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 26: this message will repeat every 5th retry.
W20260620 18:32:56.346071 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 31: this message will repeat every 5th retry.
W20260620 18:32:56.727736 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 26: this message will repeat every 5th retry.
I20260620 18:32:58.104781 13120 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:32:58.105197 13120 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:32:58.106052 13120 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:32:58.107395 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:32:58.107610 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:32:58.107798 10832 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:32:58.108140 13120 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:32:58.991487 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:32:58.991819 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:33:00.251889 13213 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:33:00.252345 13213 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:00.253899 13213 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:00.255297 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:33:00.255602 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:33:00.255738 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:33:00.256096 13213 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:33:01.918860 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 36: this message will repeat every 5th retry.
W20260620 18:33:03.247432 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 31: this message will repeat every 5th retry.
I20260620 18:33:03.996057 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:33:03.996345 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:33:04.059139 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 31: this message will repeat every 5th retry.
I20260620 18:33:06.302574 13294 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:33:06.303052 13294 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:06.303911 13294 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:06.305226 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:33:06.305563 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:33:06.305696 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:33:06.306118 13294 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:33:06.485668 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 36: this message will repeat every 5th retry.
W20260620 18:33:07.077689 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 31: this message will repeat every 5th retry.
I20260620 18:33:09.000605 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:33:09.000891 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:33:09.446394 13337 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:33:09.446695 13337 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:33:09.447548 13337 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:33:09.448909 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260620 18:33:09.450675 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:33:09.450892 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:33:09.451112 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:33:09.451504 13337 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:33:09.720109 13257 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:33:09.720458 13257 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:09.721486 13257 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:09.722770 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:33:09.723104 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:33:09.723323 10832 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:33:09.723697 13257 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:33:10.787384 13344 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:33:10.787704 13344 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:10.788642 13344 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:10.790259 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:33:10.790490 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:33:10.790593 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:33:10.790904 13344 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:33:11.683910 13363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:33:11.684253 13363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:33:11.685199 13363 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:33:11.687804 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:33:11.688056 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:33:11.688190 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:33:11.688534 13363 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:33:12.221520 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 41: this message will repeat every 5th retry.
W20260620 18:33:13.209024 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 36: this message will repeat every 5th retry.
W20260620 18:33:13.587984 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 36: this message will repeat every 5th retry.
I20260620 18:33:14.005002 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:33:14.005311 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:33:14.466341 13402 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:33:14.466614 13402 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:14.467937 13402 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:14.469342 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:33:14.469684 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:33:14.469794 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:33:14.470198 13402 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:33:14.960399 13384 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:33:14.960601 13384 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:14.961390 13384 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:14.962430 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:33:14.962656 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:33:14.962821 10832 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:33:14.963212 13384 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260620 18:33:16.151364 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 41: this message will repeat every 5th retry.
W20260620 18:33:17.152022 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 36: this message will repeat every 5th retry.
I20260620 18:33:19.009325 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:33:19.009621 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:33:20.543562 13483 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:33:20.543838 13483 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:20.544909 13483 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:20.546280 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:33:20.546583 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:33:20.546736 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:33:20.547135 13483 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:33:22.143137 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 46: this message will repeat every 5th retry.
W20260620 18:33:23.163079 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 41: this message will repeat every 5th retry.
I20260620 18:33:24.014031 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:33:24.014245 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:33:24.045737 13530 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:33:24.045979 13530 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:24.046868 13530 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:24.048496 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:33:24.048738 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:33:24.048972 10833 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:33:24.049413 13530 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260620 18:33:24.319347 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 41: this message will repeat every 5th retry.
W20260620 18:33:25.308029 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 46: this message will repeat every 5th retry.
I20260620 18:33:26.857893 13571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:33:26.858130 13571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:33:26.859125 13571 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:33:26.860877 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260620 18:33:26.862605 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:33:26.862872 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:33:26.862985 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:33:26.863390 13571 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:33:26.913357 13568 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:33:26.913704 13568 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:26.914590 13568 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:26.915848 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:33:26.916129 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:33:26.916214 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:33:26.916528 13568 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:33:27.181996 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 41: this message will repeat every 5th retry.
I20260620 18:33:29.017783 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:33:29.018033 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:33:32.998227 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 51: this message will repeat every 5th retry.
W20260620 18:33:33.332904 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 46: this message will repeat every 5th retry.
I20260620 18:33:34.021994 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:33:34.022274 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:33:34.250574 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 46: this message will repeat every 5th retry.
W20260620 18:33:35.469499 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 51: this message will repeat every 5th retry.
I20260620 18:33:36.488180 13705 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:33:36.488423 13705 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:36.489398 13705 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:36.490849 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:33:36.491181 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:33:36.491343 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:33:36.491648 13705 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:33:36.541844 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 46: this message will repeat every 5th retry.
I20260620 18:33:37.020815 13705 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:33:37.021092 13705 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:37.021895 13705 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:37.023164 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:33:37.023383 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:33:37.023483 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:33:37.023726 13705 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:33:37.666925 13697 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:33:37.667235 13697 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:37.668203 13697 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:37.669225 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:33:37.669441 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:33:37.669581 10833 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:33:37.669977 13697 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:33:39.026091 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:33:39.026301 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:33:42.353492 13736 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:33:42.353804 13736 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:42.354609 13736 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:42.356125 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:33:42.356364 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:33:42.356494 10832 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:33:42.356779 13736 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:33:42.910027 13774 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:33:42.910231 13774 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:42.910987 13774 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:42.912175 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:33:42.912504 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:33:42.912679 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:33:42.913013 13774 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:33:42.963112 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 56: this message will repeat every 5th retry.
W20260620 18:33:42.967123 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 51: this message will repeat every 5th retry.
I20260620 18:33:43.234871 13774 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:33:43.235160 13774 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:43.236047 13774 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:43.237226 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:33:43.237593 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:33:43.237722 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:33:43.238095 13774 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:33:43.376336 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 51: this message will repeat every 5th retry.
I20260620 18:33:44.030303 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:33:44.030537 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:33:45.581986 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 56: this message will repeat every 5th retry.
I20260620 18:33:46.630244 13825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:33:46.630620 13825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:33:46.631749 13825 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:33:46.633567 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260620 18:33:46.635303 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:33:46.635732 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:33:46.635879 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:33:46.636238 13825 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:33:46.927942 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 51: this message will repeat every 5th retry.
I20260620 18:33:49.034413 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:33:49.034703 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:33:50.528069 10944 heartbeater.cc:644] Failed to heartbeat to 127.9.231.252:34267 (582 consecutive failures): Network error: Failed to ping master at 127.9.231.252:34267: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:33:50.853880 10798 heartbeater.cc:644] Failed to heartbeat to 127.9.231.253:44633 (187 consecutive failures): Network error: Failed to ping master at 127.9.231.253:44633: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:33:50.988938 11089 heartbeater.cc:644] Failed to heartbeat to 127.9.231.253:44633 (189 consecutive failures): Network error: Failed to ping master at 127.9.231.253:44633: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:33:51.446911 13886 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:33:51.447168 13886 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:51.448319 13886 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:51.449558 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:33:51.449847 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:33:51.449981 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:33:51.450311 13886 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:33:51.633066 13893 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:33:51.633450 13893 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:33:51.634239 13893 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:33:51.635851 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260620 18:33:51.637547 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:33:51.637784 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:33:51.637880 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:33:51.638231 13893 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:33:51.849105 13886 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:33:51.849366 13886 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:51.850437 13886 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:51.851728 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:33:51.851967 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:33:51.852078 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:33:51.852439 13886 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:33:52.735196 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 61: this message will repeat every 5th retry.
W20260620 18:33:52.870077 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 56: this message will repeat every 5th retry.
I20260620 18:33:53.212209 13919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:33:53.212589 13919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:33:53.213424 13919 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:33:53.216463 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:33:53.216778 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:33:53.216920 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:33:53.217275 13919 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:33:53.601245 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 56: this message will repeat every 5th retry.
I20260620 18:33:53.617340 13911 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:33:53.617615 13911 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:53.618883 13911 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:53.620040 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:33:53.620309 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:33:53.620735 10832 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:33:53.621189 13911 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:33:54.038300 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:33:54.038532 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:33:55.744784 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 61: this message will repeat every 5th retry.
W20260620 18:33:56.593359 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 56: this message will repeat every 5th retry.
I20260620 18:33:59.042366 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:33:59.042634 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:33:59.352631 13974 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:33:59.352864 13974 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:33:59.353657 13974 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:33:59.355118 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:33:59.355522 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:33:59.355724 10832 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:33:59.355994 13974 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:34:01.641784 14033 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:34:01.641992 14033 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:34:01.642681 14033 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:34:01.643827 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:34:01.644134 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:34:01.644294 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:34:01.644611 14033 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:34:02.014673 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 61: this message will repeat every 5th retry.
W20260620 18:34:02.741714 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 66: this message will repeat every 5th retry.
W20260620 18:34:03.575928 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 61: this message will repeat every 5th retry.
I20260620 18:34:04.046607 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:34:04.046818 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:34:05.522214 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 66: this message will repeat every 5th retry.
W20260620 18:34:05.740145 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 61: this message will repeat every 5th retry.
I20260620 18:34:09.050875 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:34:09.051086 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:34:11.525084 14147 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:34:11.525302 14147 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:34:11.526252 14147 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:34:11.527335 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:34:11.527637 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:34:11.527750 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:34:11.528084 14147 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:34:11.742501 14147 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:34:11.742852 14147 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:34:11.743642 14147 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:34:11.745015 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:34:11.745281 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:34:11.745429 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:34:11.745779 14147 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:34:12.473073 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 66: this message will repeat every 5th retry.
I20260620 18:34:12.748344 14162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:34:12.748594 14162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:34:12.749475 14162 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:34:12.750941 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260620 18:34:12.752666 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:34:12.752904 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:34:12.753046 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:34:12.753398 14162 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:34:13.584180 14089 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:34:13.584405 14089 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:34:13.585302 14089 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:34:13.586439 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:34:13.586654 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:34:13.586889 10833 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:34:13.587301 14089 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260620 18:34:13.665398 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 71: this message will repeat every 5th retry.
W20260620 18:34:13.809437 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 66: this message will repeat every 5th retry.
I20260620 18:34:14.055137 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:34:14.055390 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:34:14.588346 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 71: this message will repeat every 5th retry.
W20260620 18:34:15.659034 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 66: this message will repeat every 5th retry.
I20260620 18:34:17.399863 14235 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:34:17.400157 14235 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:34:17.400993 14235 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:34:17.404671 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:34:17.405056 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:34:17.405198 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:34:17.405524 14235 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:34:19.005240 12515 maintenance_manager.cc:419] P 1cd216a992124f0fac2fb65754e6f7df: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033364
I20260620 18:34:19.059264 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:34:19.059521 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:34:19.059705 12514 maintenance_manager.cc:643] P 1cd216a992124f0fac2fb65754e6f7df: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.053s user 0.031s sys 0.000s Metrics: {"bytes_written":15493,"cfile_init":1,"dirs.queue_time_us":822,"dirs.run_cpu_time_us":328,"dirs.run_wall_time_us":1478,"drs_written":1,"lbm_read_time_us":76,"lbm_reads_lt_1ms":4,"lbm_write_time_us":777,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":32,"thread_start_us":720,"threads_started":2,"wal-append.queue_time_us":493}
I20260620 18:34:19.061303 12515 maintenance_manager.cc:419] P 1cd216a992124f0fac2fb65754e6f7df: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1754 bytes on disk
I20260620 18:34:19.062459 12514 maintenance_manager.cc:643] P 1cd216a992124f0fac2fb65754e6f7df: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4}
W20260620 18:34:23.146733 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 71: this message will repeat every 5th retry.
W20260620 18:34:23.277845 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 76: this message will repeat every 5th retry.
W20260620 18:34:23.294282 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 71: this message will repeat every 5th retry.
I20260620 18:34:23.557613 14303 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:34:23.557869 14303 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:34:23.558722 14303 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:34:23.560401 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:34:23.560726 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:34:23.560833 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:34:23.561276 14303 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:34:24.063191 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:34:24.063468 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:34:24.405055 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 76: this message will repeat every 5th retry.
I20260620 18:34:25.606125 14339 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:34:25.606367 14339 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:34:25.607082 14339 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:34:25.608341 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:34:25.608621 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:34:25.608752 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:34:25.609129 14339 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:34:26.364796 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 71: this message will repeat every 5th retry.
I20260620 18:34:29.067323 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:34:29.067595 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:34:29.577941 14401 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:34:29.578310 14401 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:34:29.579180 14401 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:34:29.580641 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260620 18:34:29.582223 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:34:29.582652 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:34:29.582764 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:34:29.583057 14401 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:34:32.057340 14445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:34:32.057550 14445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:34:32.058153 14445 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:34:32.062891 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:34:32.063135 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:34:32.063273 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:34:32.063623 14445 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:34:33.273577 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 76: this message will repeat every 5th retry.
W20260620 18:34:33.372583 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 81: this message will repeat every 5th retry.
I20260620 18:34:33.465766 14471 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:34:33.466033 14471 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:34:33.467193 14471 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:34:33.468676 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:34:33.468921 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:34:33.469061 10832 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:34:33.469575 14471 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260620 18:34:33.472630 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 76: this message will repeat every 5th retry.
I20260620 18:34:33.937479 14476 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:34:33.937762 14476 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:34:33.938655 14476 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:34:33.940127 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:34:33.940507 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:34:33.940599 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:34:33.940990 14476 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:34:34.072326 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:34:34.072597 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:34:34.461900 14496 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:34:34.462299 14496 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:34:34.463133 14496 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:34:34.464418 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:34:34.464619 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:34:34.464771 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:34:34.465111 14496 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:34:34.469269 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 81: this message will repeat every 5th retry.
W20260620 18:34:35.866312 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 76: this message will repeat every 5th retry.
I20260620 18:34:37.419370 14547 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:34:37.419581 14547 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:34:37.420464 14547 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:34:37.421720 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:34:37.421939 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:34:37.422019 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:34:37.422379 14547 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:34:39.077018 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:34:39.077315 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:34:40.899694 14602 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:34:40.900238 14602 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:34:40.901170 14602 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:34:40.902357 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:34:40.902649 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:34:40.902784 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:34:40.903169 14602 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:34:42.178644 14596 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:34:42.178972 14596 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:34:42.179908 14596 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:34:42.181303 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:34:42.181595 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:34:42.181766 10833 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:34:42.182018 14596 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260620 18:34:42.596653 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 81: this message will repeat every 5th retry.
W20260620 18:34:43.637228 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 81: this message will repeat every 5th retry.
W20260620 18:34:43.739583 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 86: this message will repeat every 5th retry.
I20260620 18:34:44.081241 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:34:44.081494 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:34:44.169828 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 86: this message will repeat every 5th retry.
W20260620 18:34:45.657063 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 81: this message will repeat every 5th retry.
I20260620 18:34:46.153853 14691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:34:46.154230 14691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:34:46.155205 14691 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:34:46.156833 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260620 18:34:46.162020 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:34:46.162246 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:34:46.162387 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:34:46.162793 14691 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:34:46.349098 14699 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:34:46.349363 14699 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:34:46.350234 14699 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:34:46.351766 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:34:46.352082 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:34:46.352202 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:34:46.352608 14699 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:34:49.086200 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:34:49.086508 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:34:50.982404 10944 heartbeater.cc:644] Failed to heartbeat to 127.9.231.252:34267 (915 consecutive failures): Network error: Failed to ping master at 127.9.231.252:34267: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:34:51.111526 10796 heartbeater.cc:644] Failed to heartbeat to 127.9.231.252:34267 (343 consecutive failures): Network error: Failed to ping master at 127.9.231.252:34267: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:34:51.252231 11087 heartbeater.cc:644] Failed to heartbeat to 127.9.231.252:34267 (345 consecutive failures): Network error: Failed to ping master at 127.9.231.252:34267: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:34:52.333014 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 86: this message will repeat every 5th retry.
W20260620 18:34:53.376139 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 86: this message will repeat every 5th retry.
I20260620 18:34:53.596922 14803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:34:53.597221 14803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:34:53.597966 14803 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:34:53.599629 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260620 18:34:53.601341 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:34:53.601565 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:34:53.601694 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:34:53.602080 14803 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:34:54.029438 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 91: this message will repeat every 5th retry.
W20260620 18:34:54.044840 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 91: this message will repeat every 5th retry.
I20260620 18:34:54.089998 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:34:54.090245 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:34:55.957556 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 86: this message will repeat every 5th retry.
I20260620 18:34:56.820748 14841 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:34:56.821112 14841 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:34:56.821870 14857 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:34:56.822291 14841 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
I20260620 18:34:56.822095 14857 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:34:56.823199 14857 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:34:56.823753 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:34:56.824194 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:34:56.824312 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:34:56.824388 10832 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
W20260620 18:34:56.824535 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:34:56.824682 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:34:56.824743 14841 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:34:56.825055 14857 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:34:59.094113 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:34:59.094349 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:35:01.359134 14909 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:35:01.359393 14909 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:35:01.360278 14909 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:35:01.361548 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:35:01.361905 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:35:01.362097 10833 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:35:01.362444 14909 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260620 18:35:02.056759 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 91: this message will repeat every 5th retry.
I20260620 18:35:02.431054 14941 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:35:02.431383 14941 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:35:02.432216 14941 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:35:02.433988 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260620 18:35:02.436101 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:35:02.436470 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:35:02.436661 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:35:02.437048 14941 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:35:02.730697 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 91: this message will repeat every 5th retry.
W20260620 18:35:04.078781 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 96: this message will repeat every 5th retry.
I20260620 18:35:04.098999 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:35:04.099294 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:35:04.384976 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 96: this message will repeat every 5th retry.
I20260620 18:35:05.882561 14932 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:35:05.882912 14932 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:35:05.883749 14932 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:35:05.884908 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:35:05.885309 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:35:05.885478 10832 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:35:05.885756 14932 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260620 18:35:06.135509 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 91: this message will repeat every 5th retry.
I20260620 18:35:06.296157 14976 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:35:06.296561 14976 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:35:06.297441 14976 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:35:06.299104 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:35:06.299480 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:35:06.299580 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:35:06.299985 14976 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:35:06.983362 15013 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:35:06.983664 15013 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:35:06.984675 15013 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:35:06.987818 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:35:06.988035 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:35:06.988106 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:35:06.988459 15013 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:35:09.103696 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:35:09.103961 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:35:10.370312 15067 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:35:10.370565 15067 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:35:10.371197 15067 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:35:10.372879 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260620 18:35:10.375306 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:35:10.375591 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:35:10.375703 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:35:10.376092 15067 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:35:12.352389 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 96: this message will repeat every 5th retry.
W20260620 18:35:12.375808 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 96: this message will repeat every 5th retry.
I20260620 18:35:13.452116 15116 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:35:13.452560 15116 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:35:13.453563 15116 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:35:13.455150 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:35:13.455447 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:35:13.455579 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:35:13.456120 15116 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:35:13.822528 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 101: this message will repeat every 5th retry.
W20260620 18:35:14.072047 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 101: this message will repeat every 5th retry.
I20260620 18:35:14.107924 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:35:14.108162 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:35:14.520504 15129 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:35:14.520795 15129 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:35:14.521637 15129 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:35:14.522830 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:35:14.523293 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:35:14.523445 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:35:14.523895 15129 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:35:16.192338 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 96: this message will repeat every 5th retry.
I20260620 18:35:19.111702 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:35:19.111989 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:35:19.793893 15135 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:35:19.794198 15135 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:35:19.795324 15135 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:35:19.796764 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:35:19.797006 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:35:19.797101 10833 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:35:19.797549 15135 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:35:20.014640 15218 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:35:20.014864 15218 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:35:20.015681 15218 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:35:20.017390 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:35:20.017632 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:35:20.017714 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:35:20.018031 15218 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:35:21.578310 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 101: this message will repeat every 5th retry.
W20260620 18:35:22.099898 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 101: this message will repeat every 5th retry.
I20260620 18:35:24.115706 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:35:24.115890 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:35:24.119889 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 106: this message will repeat every 5th retry.
W20260620 18:35:24.503993 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 106: this message will repeat every 5th retry.
I20260620 18:35:24.743481 15278 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:35:24.743850 15278 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:35:24.744650 15278 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:35:24.746080 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:35:24.746405 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:35:24.746536 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:35:24.746917 15278 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:35:25.909415 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 101: this message will repeat every 5th retry.
I20260620 18:35:29.119854 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:35:29.120090 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:35:29.518751 15181 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:35:29.519088 15181 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:35:29.519920 15181 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:35:29.520917 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:35:29.521083 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:35:29.521317 10833 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:35:29.521620 15181 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:35:29.786037 15356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:35:29.786365 15356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:35:29.787374 15356 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:35:29.789106 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260620 18:35:29.791294 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:35:29.791567 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:35:29.791839 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:35:29.792220 15356 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:35:31.510183 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 106: this message will repeat every 5th retry.
W20260620 18:35:31.672766 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 106: this message will repeat every 5th retry.
I20260620 18:35:32.122628 15401 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:35:32.122947 15401 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:35:32.123966 15401 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:35:32.125032 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:35:32.125217 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:35:32.125319 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:35:32.125569 15401 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:35:33.762681 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 111: this message will repeat every 5th retry.
I20260620 18:35:34.123980 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:35:34.124204 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:35:34.459172 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 111: this message will repeat every 5th retry.
W20260620 18:35:36.293682 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 106: this message will repeat every 5th retry.
I20260620 18:35:38.853480 15491 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:35:38.853763 15491 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:35:38.854545 15491 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:35:38.855924 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:35:38.856205 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:35:38.856346 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:35:38.856696 15491 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:35:39.128587 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:35:39.128912 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:35:39.915691 15523 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:35:39.915962 15523 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:35:39.916721 15523 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:35:39.918145 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260620 18:35:39.919771 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:35:39.919979 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:35:39.920071 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:35:39.920361 15523 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:35:41.434366 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 111: this message will repeat every 5th retry.
W20260620 18:35:41.706185 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 111: this message will repeat every 5th retry.
W20260620 18:35:42.921648 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 116: this message will repeat every 5th retry.
I20260620 18:35:43.016120 15575 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:35:43.016512 15575 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:35:43.017525 15575 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:35:43.020372 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:35:43.020642 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:35:43.020748 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:35:43.021023 15575 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:35:43.021749 15574 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:35:43.022084 15574 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:35:43.022902 15574 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:35:43.024214 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:35:43.024577 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:35:43.024720 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:35:43.025072 15574 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:35:43.765864 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 116: this message will repeat every 5th retry.
I20260620 18:35:44.133028 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:35:44.133329 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:35:45.654614 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 111: this message will repeat every 5th retry.
I20260620 18:35:45.882185 15517 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:35:45.882419 15517 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:35:45.883400 15517 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:35:45.884622 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:35:45.884922 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:35:45.885119 10833 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:35:45.885681 15517 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:35:49.138051 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:35:49.138281 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:35:49.787706 15671 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:35:49.787976 15671 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:35:49.788949 15671 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:35:49.790584 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260620 18:35:49.792306 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:35:49.792538 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:35:49.792652 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:35:49.792948 15671 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:35:51.316797 10944 heartbeater.cc:644] Failed to heartbeat to 127.9.231.252:34267 (1239 consecutive failures): Network error: Failed to ping master at 127.9.231.252:34267: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:35:51.351514 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 116: this message will repeat every 5th retry.
W20260620 18:35:51.394363 10796 heartbeater.cc:644] Failed to heartbeat to 127.9.231.252:34267 (463 consecutive failures): Network error: Failed to ping master at 127.9.231.252:34267: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:35:51.394363 10798 heartbeater.cc:644] Failed to heartbeat to 127.9.231.253:44633 (427 consecutive failures): Network error: Failed to ping master at 127.9.231.253:44633: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:35:51.535157 11087 heartbeater.cc:644] Failed to heartbeat to 127.9.231.252:34267 (465 consecutive failures): Network error: Failed to ping master at 127.9.231.252:34267: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:35:51.931107 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 116: this message will repeat every 5th retry.
I20260620 18:35:52.546532 15719 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:35:52.546845 15719 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:35:52.547930 15719 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:35:52.549261 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:35:52.549544 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:35:52.549624 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:35:52.549897 15719 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:35:53.279472 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 121: this message will repeat every 5th retry.
I20260620 18:35:53.324026 15719 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:35:53.324362 15719 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:35:53.325054 15719 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:35:53.326570 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:35:53.326785 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:35:53.326901 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:35:53.327333 15719 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:35:53.727509 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 121: this message will repeat every 5th retry.
I20260620 18:35:54.142699 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:35:54.142926 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:35:55.299597 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 116: this message will repeat every 5th retry.
I20260620 18:35:58.101295 15789 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:35:58.101516 15789 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:35:58.102252 15789 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:35:58.104887 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260620 18:35:58.107065 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:35:58.107343 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:35:58.107522 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:35:58.108635 15789 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:35:58.670316 15769 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:35:58.670589 15769 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:35:58.671514 15769 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:35:58.672628 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:35:58.672886 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:35:58.672981 10833 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:35:58.673261 15769 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:35:59.147928 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:35:59.148227 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:36:01.379896 15847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:36:01.380110 15847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:36:01.380908 15847 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:36:01.386005 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:36:01.386337 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:36:01.386461 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:36:01.386831 15847 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:36:01.734719 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 121: this message will repeat every 5th retry.
I20260620 18:36:01.922926 15837 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:36:01.923256 15837 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:36:01.924368 15837 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:36:01.925654 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:36:01.925890 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:36:01.926082 10832 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:36:01.926503 15837 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260620 18:36:02.161824 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 121: this message will repeat every 5th retry.
I20260620 18:36:02.419099 15862 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:36:02.419378 15862 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:36:02.420392 15862 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:36:02.421958 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:36:02.422302 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:36:02.422459 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:36:02.422863 15862 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:36:03.411672 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 126: this message will repeat every 5th retry.
W20260620 18:36:04.062862 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 126: this message will repeat every 5th retry.
I20260620 18:36:04.152503 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:36:04.152886 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:36:05.296962 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 121: this message will repeat every 5th retry.
I20260620 18:36:05.802765 15917 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:36:05.803313 15917 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:36:05.804164 15917 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:36:05.805393 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:36:05.805657 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:36:05.805791 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:36:05.806092 15917 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:36:05.910979 15917 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:36:05.911314 15917 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:36:05.912204 15917 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:36:05.913524 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:36:05.913828 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:36:05.914373 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:36:05.914776 15917 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:36:09.157207 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:36:09.157444 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:36:10.524461 15985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:36:10.524770 15985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:36:10.525605 15985 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:36:10.527165 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260620 18:36:10.529143 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:36:10.529363 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:36:10.529443 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:36:10.529822 15985 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:36:11.732903 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 126: this message will repeat every 5th retry.
W20260620 18:36:12.579608 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 126: this message will repeat every 5th retry.
W20260620 18:36:13.180311 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 131: this message will repeat every 5th retry.
W20260620 18:36:13.711071 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 131: this message will repeat every 5th retry.
I20260620 18:36:14.161208 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:36:14.161442 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:36:14.843160 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 126: this message will repeat every 5th retry.
I20260620 18:36:17.221886 16094 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:36:17.222222 16094 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:36:17.223448 16094 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:36:17.225046 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:36:17.225306 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:36:17.225452 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:36:17.225858 16094 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:36:17.375778 16101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:36:17.375986 16101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:36:17.376797 16101 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:36:17.378566 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260620 18:36:17.380573 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:36:17.380874 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:36:17.380987 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:36:17.381313 16101 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:36:19.165809 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:36:19.166054 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:36:20.109722 16145 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:36:20.110034 16145 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:36:20.110834 16145 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:36:20.112315 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:36:20.112672 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:36:20.112795 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:36:20.113216 16145 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:36:21.580561 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 131: this message will repeat every 5th retry.
I20260620 18:36:21.904564 16167 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:36:21.904906 16167 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:36:21.906101 16167 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:36:21.907315 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:36:21.907503 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:36:21.907610 10833 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:36:21.907912 16167 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:36:22.196107 16179 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:36:22.196442 16179 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:36:22.197234 16179 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:36:22.200685 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:36:22.200969 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:36:22.201117 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:36:22.201469 16179 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:36:22.384882 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 131: this message will repeat every 5th retry.
W20260620 18:36:23.222776 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 136: this message will repeat every 5th retry.
W20260620 18:36:24.024649 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 136: this message will repeat every 5th retry.
I20260620 18:36:24.170516 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:36:24.170785 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:36:24.381278 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 131: this message will repeat every 5th retry.
I20260620 18:36:27.440964 16257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:36:27.441227 16257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:36:27.442027 16257 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:36:27.443679 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260620 18:36:27.445350 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:36:27.445595 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:36:27.445739 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:36:27.446074 16257 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:36:28.557883 16276 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:36:28.558137 16276 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:36:28.558897 16276 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:36:28.559901 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:36:28.560205 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:36:28.560389 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:36:28.560664 16276 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:36:29.174882 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:36:29.175144 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:36:29.680485 16294 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:36:29.681169 16294 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:36:29.682116 16294 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:36:29.685560 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:36:29.685820 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:36:29.685981 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:36:29.686331 16294 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:36:30.365509 16305 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:36:30.365828 16305 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:36:30.366670 16305 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:36:30.368134 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:36:30.368431 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:36:30.368522 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:36:30.368847 16305 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:36:31.283797 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 136: this message will repeat every 5th retry.
I20260620 18:36:31.665606 16328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:36:31.666148 16328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:36:31.667166 16328 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:36:31.670260 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:36:31.670492 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
I20260620 18:36:31.670629 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:36:31.670990 16328 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:36:32.397895 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 136: this message will repeat every 5th retry.
W20260620 18:36:33.323091 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 141: this message will repeat every 5th retry.
W20260620 18:36:33.550422 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 141: this message will repeat every 5th retry.
W20260620 18:36:34.089251 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 136: this message will repeat every 5th retry.
I20260620 18:36:34.179488 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:36:34.179800 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:36:37.159950 16407 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:36:37.160305 16407 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:36:37.161201 16407 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:36:37.162731 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:36:37.163049 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:36:37.163193 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:36:37.163548 16407 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:36:37.183122 16395 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:36:37.183385 16395 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:36:37.184435 16395 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:36:37.185595 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:36:37.185792 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:36:37.186048 10833 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:36:37.186355 16395 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:36:39.184034 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:36:39.184386 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:36:40.318078 16447 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:36:40.318416 16447 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:36:40.319164 16447 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:36:40.320569 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:36:40.321012 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:36:40.321221 10832 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:36:40.321659 16447 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260620 18:36:41.465350 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 141: this message will repeat every 5th retry.
I20260620 18:36:41.561983 16473 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:36:41.562247 16473 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:36:41.562927 16473 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:36:41.564374 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:36:41.564705 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:36:41.564821 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:36:41.565207 16473 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:36:41.723569 16473 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:36:41.723852 16473 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:36:41.724542 16473 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:36:41.726013 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:36:41.726307 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:36:41.726441 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:36:41.726846 16473 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:36:41.859498 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 141: this message will repeat every 5th retry.
W20260620 18:36:43.071635 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 146: this message will repeat every 5th retry.
W20260620 18:36:43.249128 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 146: this message will repeat every 5th retry.
I20260620 18:36:44.188154 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:36:44.188465 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:36:44.516230 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 141: this message will repeat every 5th retry.
I20260620 18:36:45.625144 16532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:36:45.625353 16532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:36:45.625991 16532 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:36:45.627745 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260620 18:36:45.630247 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:36:45.630517 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:36:45.630741 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:36:45.631404 16532 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:36:46.648669 16466 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:36:46.648869 16466 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:36:46.649516 16466 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:36:46.650738 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:36:46.650971 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:36:46.651109 10833 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:36:46.651379 16466 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:36:49.192801 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:36:49.193219 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:36:51.350787 10946 heartbeater.cc:644] Failed to heartbeat to 127.9.231.253:44633 (1482 consecutive failures): Network error: Failed to ping master at 127.9.231.253:44633: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:36:51.662137 10798 heartbeater.cc:644] Failed to heartbeat to 127.9.231.253:44633 (547 consecutive failures): Network error: Failed to ping master at 127.9.231.253:44633: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:36:51.662127 10796 heartbeater.cc:644] Failed to heartbeat to 127.9.231.252:34267 (583 consecutive failures): Network error: Failed to ping master at 127.9.231.252:34267: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:36:51.824512 11087 heartbeater.cc:644] Failed to heartbeat to 127.9.231.252:34267 (585 consecutive failures): Network error: Failed to ping master at 127.9.231.252:34267: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:36:51.852118 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 146: this message will repeat every 5th retry.
W20260620 18:36:52.630048 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 146: this message will repeat every 5th retry.
W20260620 18:36:53.034668 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 151: this message will repeat every 5th retry.
W20260620 18:36:53.223078 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 151: this message will repeat every 5th retry.
W20260620 18:36:53.629637 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 146: this message will repeat every 5th retry.
I20260620 18:36:54.197252 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:36:54.197481 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:36:56.267905 16573 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:36:56.268148 16573 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:36:56.269148 16573 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:36:56.270149 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:36:56.270613 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:36:56.270799 10833 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:36:56.271178 16573 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:36:59.201238 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:36:59.201556 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:36:59.427845 16727 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:36:59.428200 16727 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:36:59.429236 16727 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:36:59.430806 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:36:59.431058 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:36:59.431146 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:36:59.431558 16727 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:37:00.297374 16740 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:37:00.297628 16740 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:37:00.298527 16740 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:37:00.299813 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:37:00.300021 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:37:00.300086 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:37:00.300377 16740 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260620 18:37:01.290305 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 151: this message will repeat every 5th retry.
W20260620 18:37:02.683482 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 151: this message will repeat every 5th retry.
W20260620 18:37:02.922099 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 156: this message will repeat every 5th retry.
W20260620 18:37:03.037168 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 156: this message will repeat every 5th retry.
W20260620 18:37:03.347808 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 151: this message will repeat every 5th retry.
I20260620 18:37:04.205888 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:37:04.206166 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:37:04.323196 16800 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:37:04.323618 16800 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:37:04.324673 16800 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:37:04.325860 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260620 18:37:04.327777 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:37:04.327965 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:37:04.328047 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:37:04.328346 16800 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:37:08.696182 16873 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:37:08.696502 16873 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:37:08.697127 16873 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:37:08.699962 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:37:08.700189 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:37:08.700331 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:37:08.700623 16873 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:37:09.209914 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:37:09.210248 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:37:09.344200 16880 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:37:09.344517 16880 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:37:09.345301 16880 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:37:09.346457 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:37:09.346688 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:37:09.346858 10832 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:37:09.347148 16880 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:37:11.366355 16918 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:37:11.366631 16918 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:37:11.367658 16918 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:37:11.368719 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:37:11.368966 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:37:11.369107 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:37:11.369457 16918 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:37:12.169991 16932 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:37:12.170281 16932 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:37:12.171202 16932 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:37:12.172545 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260620 18:37:12.174144 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:37:12.174458 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:37:12.174549 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:37:12.174901 16932 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260620 18:37:12.334086 10833 consensus_peers.cc:487] T 1709fe2d11a24f52bd1812bc37e1714b P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1709fe2d11a24f52bd1812bc37e1714b. This is attempt 156: this message will repeat every 5th retry.
W20260620 18:37:12.421555 10832 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 161: this message will repeat every 5th retry.
W20260620 18:37:12.862638 10833 consensus_peers.cc:487] T 4cdd0fd6fcb842239d0fb74758cd1290 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cdd0fd6fcb842239d0fb74758cd1290. This is attempt 156: this message will repeat every 5th retry.
W20260620 18:37:13.037832 10832 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Couldn't send request to peer 7696506dcc9746d0aea9b1129a3e9433. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 161: this message will repeat every 5th retry.
W20260620 18:37:13.589404 10833 consensus_peers.cc:487] T b7a80fba1822404a98459c47faef4680 P a0ae894b0758446aa6219aaa5267d1b4 -> Peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Couldn't send request to peer 9a4ce671d3984e1a8592bc29d45612bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a80fba1822404a98459c47faef4680. This is attempt 156: this message will repeat every 5th retry.
I20260620 18:37:13.806617 16895 raft_consensus.cc:482] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9a4ce671d3984e1a8592bc29d45612bc)
I20260620 18:37:13.806954 16895 raft_consensus.cc:504] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:37:13.808020 16895 leader_election.cc:288] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:37:13.809254 10832 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7696506dcc9746d0aea9b1129a3e9433 (127.9.231.193:41961): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
W20260620 18:37:13.809475 10833 leader_election.cc:341] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8009d3fea30b4efa951e07d0bc2da1a5
I20260620 18:37:13.809660 10833 leader_election.cc:302] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0ae894b0758446aa6219aaa5267d1b4; no voters: 7696506dcc9746d0aea9b1129a3e9433, 9a4ce671d3984e1a8592bc29d45612bc
I20260620 18:37:13.810027 16895 raft_consensus.cc:2726] T 8009d3fea30b4efa951e07d0bc2da1a5 P a0ae894b0758446aa6219aaa5267d1b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:37:14.213788 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:37:14.214005 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:37:16.391955 16993 raft_consensus.cc:482] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:37:16.392170 16993 raft_consensus.cc:504] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:37:16.393047 16993 leader_election.cc:288] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:37:16.394312 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
W20260620 18:37:16.394567 10687 leader_election.cc:341] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: 8b82c74e9aa6451c8e7c537f7096aa25
I20260620 18:37:16.394661 10687 leader_election.cc:302] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:37:16.394984 16993 raft_consensus.cc:2726] T 8b82c74e9aa6451c8e7c537f7096aa25 P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:37:18.487097 16993 raft_consensus.cc:482] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:37:18.487319 16993 raft_consensus.cc:504] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7696506dcc9746d0aea9b1129a3e9433" member_type: VOTER last_known_addr { host: "127.9.231.193" port: 41961 } } peers { permanent_uuid: "a0ae894b0758446aa6219aaa5267d1b4" member_type: VOTER last_known_addr { host: "127.9.231.194" port: 45409 } } peers { permanent_uuid: "9a4ce671d3984e1a8592bc29d45612bc" member_type: VOTER last_known_addr { host: "127.9.231.195" port: 42583 } }
I20260620 18:37:18.487946 16993 leader_election.cc:288] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409), 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583)
W20260620 18:37:18.489187 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0ae894b0758446aa6219aaa5267d1b4 (127.9.231.194:45409): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
W20260620 18:37:18.489388 10687 leader_election.cc:341] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a4ce671d3984e1a8592bc29d45612bc (127.9.231.195:42583): Not found: Tablet not found: acc2a8b335d14d20bad1719566a27f3d
I20260620 18:37:18.489471 10687 leader_election.cc:302] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7696506dcc9746d0aea9b1129a3e9433; no voters: 9a4ce671d3984e1a8592bc29d45612bc, a0ae894b0758446aa6219aaa5267d1b4
I20260620 18:37:18.489718 16993 raft_consensus.cc:2726] T acc2a8b335d14d20bad1719566a27f3d P 7696506dcc9746d0aea9b1129a3e9433 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:37:19.218164 12494 master_runner.cc:186] Error getting master registration for 127.9.231.253:44633: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111),
I20260620 18:37:19.218516 12494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:37:19.226269 17042 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 650e0197335644718cd91d5018399ee2)
I20260620 18:37:19.226475 17042 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd216a992124f0fac2fb65754e6f7df" member_type: VOTER last_known_addr { host: "127.9.231.254" port: 43203 } } peers { permanent_uuid: "650e0197335644718cd91d5018399ee2" member_type: VOTER last_known_addr { host: "127.9.231.253" port: 44633 } } peers { permanent_uuid: "4587918520d849eaad95e35037e36608" member_type: VOTER last_known_addr { host: "127.9.231.252" port: 34267 } }
I20260620 18:37:19.227145 17042 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633), 4587918520d849eaad95e35037e36608 (127.9.231.252:34267)
W20260620 18:37:19.228444 12512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260620 18:37:19.230216 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 650e0197335644718cd91d5018399ee2 (127.9.231.253:44633): Network error: Client connection negotiation failed: client connection to 127.9.231.253:44633: connect: Connection refused (error 111)
W20260620 18:37:19.230476 12512 leader_election.cc:334] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4587918520d849eaad95e35037e36608 (127.9.231.252:34267): Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
I20260620 18:37:19.230603 12512 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd216a992124f0fac2fb65754e6f7df; no voters: 4587918520d849eaad95e35037e36608, 650e0197335644718cd91d5018399ee2
I20260620 18:37:19.230931 17042 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cd216a992124f0fac2fb65754e6f7df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260620 18:37:19.916850 12515 maintenance_manager.cc:419] P 1cd216a992124f0fac2fb65754e6f7df: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1754 bytes on disk
I20260620 18:37:19.922139 12514 maintenance_manager.cc:643] P 1cd216a992124f0fac2fb65754e6f7df: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.005s sys 0.000s Metrics: {"dirs.queue_time_us":363,"dirs.run_cpu_time_us":96,"dirs.run_wall_time_us":96,"thread_start_us":569,"threads_started":2,"wal-append.queue_time_us":486}
W20260620 18:37:20.081526 11099 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:37:20.081887 11099 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:37:20.082000 11099 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
W20260620 18:37:20.082113 11099 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
F20260620 18:37:20.093370 11116 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
*** Aborted at 1781980640 (unix time) try "date -d @1781980640" if you are using GNU date ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11128 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11128 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.095079 11108 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_08795a50aec84f85bc35ae8025674185 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11128 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.095079 11108 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_08795a50aec84f85bc35ae8025674185 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097347 11105 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11128 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.095079 11108 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_08795a50aec84f85bc35ae8025674185 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097347 11105 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097505 11113 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aa7e29fa2b8e4974a8ece0e46aba222a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11128 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.095079 11108 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_08795a50aec84f85bc35ae8025674185 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097347 11105 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097505 11113 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aa7e29fa2b8e4974a8ece0e46aba222a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097668 11106 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fce67ec520194c7fabdca1fce51af8a1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11128 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.095079 11108 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_08795a50aec84f85bc35ae8025674185 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097347 11105 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097505 11113 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aa7e29fa2b8e4974a8ece0e46aba222a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097668 11106 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fce67ec520194c7fabdca1fce51af8a1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099069 11118 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099206 1*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11128 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.095079 11108 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_08795a50aec84f85bc35ae8025674185 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097347 11105 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097505 11113 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aa7e29fa2b8e4974a8ece0e46aba222a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097668 11106 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fce67ec520194c7fabdca1fce51af8a1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099069 11118 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099206 11109 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5854ef7ddc554da59e75ed6ff18718ed on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11128 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.095079 11108 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_08795a50aec84f85bc35ae8025674185 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097347 11105 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097505 11113 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aa7e29fa2b8e4974a8ece0e46aba222a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097668 11106 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fce67ec520194c7fabdca1fce51af8a1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099069 11118 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099206 11109 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5854ef7ddc554da59e75ed6ff18718ed on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099786 11127 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11128 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.095079 11108 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_08795a50aec84f85bc35ae8025674185 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097347 11105 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097505 11113 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aa7e29fa2b8e4974a8ece0e46aba222a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097668 11106 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fce67ec520194c7fabdca1fce51af8a1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099069 11118 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099206 11109 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5854ef7ddc554da59e75ed6ff18718ed on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099786 11127 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099902 11115 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11128 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.095079 11108 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_08795a50aec84f85bc35ae8025674185 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097347 11105 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097505 11113 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aa7e29fa2b8e4974a8ece0e46aba222a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097668 11106 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fce67ec520194c7fabdca1fce51af8a1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099069 11118 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099206 11109 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5854ef7ddc554da59e75ed6ff18718ed on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099786 11127 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099902 11115 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.100692 11119 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260620 18:37:20.100777 Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)11120 master-stress-test.cc:352] Check failed: _s.ok() Bad status:
*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11128 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.095079 11108 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_08795a50aec84f85bc35ae8025674185 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097347 11105 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097505 11113 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aa7e29fa2b8e4974a8ece0e46aba222a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097668 11106 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fce67ec520194c7fabdca1fce51af8a1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099069 11118 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099206 11109 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5854ef7ddc554da59e75ed6ff18718ed on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099786 11127 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099902 11115 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.100692 11119 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260620 18:37:20.100777 Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)11120 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e80000279f) received by PID 10143 (TID 0x7f7340bd9700) from PID 10143; stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11128 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.095079 11108 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_08795a50aec84f85bc35ae8025674185 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097347 11105 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097505 11113 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aa7e29fa2b8e4974a8ece0e46aba222a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097668 11106 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fce67ec520194c7fabdca1fce51af8a1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099069 11118 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099206 11109 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5854ef7ddc554da59e75ed6ff18718ed on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099786 11127 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099902 11115 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.100692 11119 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260620 18:37:20.100777 Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)11120 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.102404 11129 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11128 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.095079 11108 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_08795a50aec84f85bc35ae8025674185 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097347 11105 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097505 11113 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aa7e29fa2b8e4974a8ece0e46aba222a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097668 11106 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fce67ec520194c7fabdca1fce51af8a1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099069 11118 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099206 11109 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5854ef7ddc554da59e75ed6ff18718ed on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099786 11127 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099902 11115 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.100692 11119 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260620 18:37:20.100777 Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)11120 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.102404 11129 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.102665 11114 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1ba08641833a48838fae05c1f72333e8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11128 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.095079 11108 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_08795a50aec84f85bc35ae8025674185 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097347 11105 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097505 11113 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aa7e29fa2b8e4974a8ece0e46aba222a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097668 11106 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fce67ec520194c7fabdca1fce51af8a1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099069 11118 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099206 11109 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5854ef7ddc554da59e75ed6ff18718ed on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099786 11127 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099902 11115 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.100692 11119 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260620 18:37:20.100777 Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)11120 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.102404 11129 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.102665 11114 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1ba08641833a48838fae05c1f72333e8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.103191 11126 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11128 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.095079 11108 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_08795a50aec84f85bc35ae8025674185 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097347 11105 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097505 11113 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aa7e29fa2b8e4974a8ece0e46aba222a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097668 11106 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fce67ec520194c7fabdca1fce51af8a1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099069 11118 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099206 11109 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5854ef7ddc554da59e75ed6ff18718ed on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099786 11127 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099902 11115 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.100692 11119 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260620 18:37:20.100777 Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)11120 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.102404 11129 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.102665 11114 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1ba08641833a48838fae05c1f72333e8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.103191 11126 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.103317 11107 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1ad292ef5e24a628dcf2d699e127e57 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
@ 0x7f7354190a46 google::(anonymous namespace)::FailureSignalHandler() at ??:0
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11128 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.095079 11108 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_08795a50aec84f85bc35ae8025674185 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097347 11105 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097505 11113 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aa7e29fa2b8e4974a8ece0e46aba222a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097668 11106 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fce67ec520194c7fabdca1fce51af8a1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099069 11118 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099206 11109 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5854ef7ddc554da59e75ed6ff18718ed on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099786 11127 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099902 11115 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.100692 11119 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260620 18:37:20.100777 Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)11120 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.102404 11129 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.102665 11114 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1ba08641833a48838fae05c1f72333e8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.103191 11126 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.103317 11107 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1ad292ef5e24a628dcf2d699e127e57 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.103551 11121 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F000F2026202606 20 18 : 37: : 20 .093403 11110 master-stress-test.cc: 329 ] Check failed: _s.ok() Bad status : Timed out: Error creating table default.Table_e4c5c6935f7a48cebd0f828771b1fe01 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094699 11111 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e001ba7ef7b44f09b6262dc3d55d0df0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11125 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.094877 11128 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.095079 11108 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_08795a50aec84f85bc35ae8025674185 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097347 11105 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f5de66a2d0aa456d8dcb032c03ee3fd6 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097505 11113 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aa7e29fa2b8e4974a8ece0e46aba222a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.097668 11106 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fce67ec520194c7fabdca1fce51af8a1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099069 11118 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099206 11109 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5854ef7ddc554da59e75ed6ff18718ed on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099786 11127 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.099902 11115 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.100692 11119 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260620 18:37:20.100777 Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)11120 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.102404 11129 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.102665 11114 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1ba08641833a48838fae05c1f72333e8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.103191 11126 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.103317 11107 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1ad292ef5e24a628dcf2d699e127e57 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.103551 11121 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)F20260620 18:37:20.104038 11112 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_16d5e29e7ed8458180884abae667bf9d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.9.231.254:43203,127.9.231.253:44633,127.9.231.252:34267, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.9.231.252:34267: connect: Connection refused (error 111)
*** Check failure stack trace: ***
@ 0x7f73582d2980 (unknown) at ??:0
@ 0x7f73514bee87 gsignal at ??:0
@ 0x7f73514c07f1 abort at ??:0
@ 0x7f73541829ad google::LogMessage::Fail() at ??:0
@ 0x7f735418588a google::LogMessage::SendToLog() at ??:0
@ 0x7f73541824de google::LogMessage::Flush() at ??:0
@ 0x7f7354183a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x404bb0 kudu::MasterStressTest::AlterTableThread() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/master-stress-test.cc:329
@ 0x444b44 std::__invoke_impl<>() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/bits/invoke.h:73
@ 0x4449fd std::__invoke<>() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/bits/invoke.h:95
@ 0x4449ba _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:244
@ 0x44496e std::thread::_Invoker<>::operator()() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:253
@ 0x4446a9 std::thread::_State_impl<>::_M_run() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:196
@ 0x7f735cc0ea4f execute_native_thread_routine at ??:0
@ 0x7f73582c76db start_thread at ??:0
@ 0x7f73515a161f clone at ??:0