Note: This is test shard 3 of 3.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/2
2026-09-25T19:59:57Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-09-25T19:59:57Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskg5SBXY/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskg5SBXY/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/security.properties
2026-09-25 20:00:05: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskg5SBXY/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskg5SBXY/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260925 20:00:16.374212 3871 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
/tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.254:41289
--webserver_interface=127.3.199.254
--webserver_port=0
--builtin_ntp_servers=127.3.199.212:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260925 20:00:16.716373 4158 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 20:00:16.716794 4158 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 20:00:16.716936 4158 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 20:00:16.725673 4158 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 20:00:16.725831 4158 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 20:00:16.725939 4158 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 20:00:16.726027 4158 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 20:00:16.736505 4158 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.199.212:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.254:41289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.199.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 25 Sep 2026 19:43:21 UTC on bdcb31816ec0
build id 13997
ASAN enabled
I20260925 20:00:16.738306 4158 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 20:00:16.740303 4158 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 20:00:16.749168 4163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:16.749064 4164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:16.749853 4166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 20:00:16.750262 4158 server_base.cc:1061] running on GCE node
I20260925 20:00:16.751192 4158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 20:00:16.752782 4158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 20:00:16.754014 4158 hybrid_clock.cc:648] HybridClock initialized: now 1790366416753964 us; error 37 us; skew 500 ppm
I20260925 20:00:16.754441 4158 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 20:00:16.756897 4158 webserver.cc:533] Webserver started at http://127.3.199.254:32859/ using document root <none> and password file <none>
I20260925 20:00:16.757668 4158 fs_manager.cc:362] Metadata directory not provided
I20260925 20:00:16.757797 4158 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 20:00:16.758176 4158 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 20:00:16.760948 4158 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data/instance:
uuid: "169b6de20560440b99cc08af549ce477"
format_stamp: "Formatted at 2026-09-25 20:00:16 on dist-test-slave-9gwk"
I20260925 20:00:16.761914 4158 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal/instance:
uuid: "169b6de20560440b99cc08af549ce477"
format_stamp: "Formatted at 2026-09-25 20:00:16 on dist-test-slave-9gwk"
I20260925 20:00:16.767489 4158 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 20:00:16.771229 4172 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:16.772879 4158 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 20:00:16.773082 4158 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data,/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
uuid: "169b6de20560440b99cc08af549ce477"
format_stamp: "Formatted at 2026-09-25 20:00:16 on dist-test-slave-9gwk"
I20260925 20:00:16.773298 4158 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 20:00:16.795462 4158 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 20:00:16.796434 4158 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 20:00:16.796730 4158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 20:00:16.818239 4158 rpc_server.cc:307] RPC server started. Bound to: 127.3.199.254:41289
I20260925 20:00:16.818259 4223 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.199.254:41289 every 8 connection(s)
I20260925 20:00:16.820003 4158 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data/info.pb
I20260925 20:00:16.823628 3871 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu as pid 4158
I20260925 20:00:16.823930 3871 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal/instance
I20260925 20:00:16.824648 4224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:16.826620 3871 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
/tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.253:41897
--webserver_interface=127.3.199.253
--webserver_port=0
--builtin_ntp_servers=127.3.199.212:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 20:00:16.830722 4224 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } has no permanent_uuid. Determining permanent_uuid...
I20260925 20:00:16.850319 4224 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } has no permanent_uuid. Determining permanent_uuid...
W20260925 20:00:16.851676 4175 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:16.853614 4224 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:16.903014 4224 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } attempt: 1
W20260925 20:00:16.905529 4224 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:16.973822 4224 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } attempt: 2
W20260925 20:00:16.976191 4224 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:17.067620 4224 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } attempt: 3
W20260925 20:00:17.070174 4224 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:17.188432 4225 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 20:00:17.188856 4225 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 20:00:17.188995 4225 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 20:00:17.197800 4225 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 20:00:17.197914 4225 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 20:00:17.197971 4225 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 20:00:17.198011 4225 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 20:00:17.208515 4225 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.199.212:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.253:41897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.199.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 25 Sep 2026 19:43:21 UTC on bdcb31816ec0
build id 13997
ASAN enabled
I20260925 20:00:17.210193 4225 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 20:00:17.212049 4225 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260925 20:00:17.213502 4224 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } attempt: 4
W20260925 20:00:17.215708 4224 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:17.221457 4236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:17.221529 4233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:17.221547 4234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 20:00:17.223006 4225 server_base.cc:1061] running on GCE node
I20260925 20:00:17.223913 4225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 20:00:17.225586 4225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 20:00:17.226877 4225 hybrid_clock.cc:648] HybridClock initialized: now 1790366417226821 us; error 50 us; skew 500 ppm
I20260925 20:00:17.227348 4225 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 20:00:17.230082 4225 webserver.cc:533] Webserver started at http://127.3.199.253:41541/ using document root <none> and password file <none>
I20260925 20:00:17.231042 4225 fs_manager.cc:362] Metadata directory not provided
I20260925 20:00:17.231204 4225 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 20:00:17.231626 4225 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 20:00:17.234441 4225 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/instance:
uuid: "da29ee550f82456fa7c5ce7ea916909f"
format_stamp: "Formatted at 2026-09-25 20:00:17 on dist-test-slave-9gwk"
I20260925 20:00:17.235351 4225 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal/instance:
uuid: "da29ee550f82456fa7c5ce7ea916909f"
format_stamp: "Formatted at 2026-09-25 20:00:17 on dist-test-slave-9gwk"
I20260925 20:00:17.241539 4225 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260925 20:00:17.245270 4242 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:17.246724 4225 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 20:00:17.246922 4225 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data,/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
uuid: "da29ee550f82456fa7c5ce7ea916909f"
format_stamp: "Formatted at 2026-09-25 20:00:17 on dist-test-slave-9gwk"
I20260925 20:00:17.247157 4225 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 20:00:17.267666 4225 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 20:00:17.268687 4225 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 20:00:17.268997 4225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 20:00:17.291000 4225 rpc_server.cc:307] RPC server started. Bound to: 127.3.199.253:41897
I20260925 20:00:17.291020 4293 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.199.253:41897 every 8 connection(s)
I20260925 20:00:17.292730 4225 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
I20260925 20:00:17.296162 3871 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu as pid 4225
I20260925 20:00:17.296397 3871 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal/instance
I20260925 20:00:17.297623 4294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:17.297786 3871 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
/tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.252:37957
--webserver_interface=127.3.199.252
--webserver_port=0
--builtin_ntp_servers=127.3.199.212:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 20:00:17.305133 4294 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } has no permanent_uuid. Determining permanent_uuid...
I20260925 20:00:17.318845 4294 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } has no permanent_uuid. Determining permanent_uuid...
I20260925 20:00:17.328038 4294 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } has no permanent_uuid. Determining permanent_uuid...
W20260925 20:00:17.329507 4246 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:17.331137 4294 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.199.252:37957: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:17.380479 4294 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } attempt: 1
W20260925 20:00:17.382840 4294 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.199.252:37957: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:17.451153 4294 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } attempt: 2
W20260925 20:00:17.453349 4294 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.199.252:37957: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:17.514952 4224 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } attempt: 5
I20260925 20:00:17.523823 4224 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } has no permanent_uuid. Determining permanent_uuid...
W20260925 20:00:17.525921 4224 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.199.252:37957: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:17.544633 4294 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } attempt: 3
W20260925 20:00:17.546939 4294 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.199.252:37957: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:17.577163 4224 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } attempt: 1
W20260925 20:00:17.579486 4224 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.199.252:37957: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:17.647768 4224 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } attempt: 2
W20260925 20:00:17.650166 4224 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.199.252:37957: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:17.661012 4295 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 20:00:17.661419 4295 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 20:00:17.661551 4295 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 20:00:17.670290 4295 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 20:00:17.670435 4295 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 20:00:17.670552 4295 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 20:00:17.670634 4295 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 20:00:17.681403 4295 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.199.212:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.252:37957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.199.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 25 Sep 2026 19:43:21 UTC on bdcb31816ec0
build id 13997
ASAN enabled
I20260925 20:00:17.683133 4295 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 20:00:17.685060 4295 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260925 20:00:17.690230 4294 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } attempt: 4
W20260925 20:00:17.692898 4294 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.199.252:37957: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:17.693614 4303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:17.693872 4304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:17.694474 4306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 20:00:17.695663 4295 server_base.cc:1061] running on GCE node
I20260925 20:00:17.696569 4295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 20:00:17.697989 4295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 20:00:17.699200 4295 hybrid_clock.cc:648] HybridClock initialized: now 1790366417699160 us; error 30 us; skew 500 ppm
I20260925 20:00:17.699594 4295 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 20:00:17.701889 4295 webserver.cc:533] Webserver started at http://127.3.199.252:32867/ using document root <none> and password file <none>
I20260925 20:00:17.702643 4295 fs_manager.cc:362] Metadata directory not provided
I20260925 20:00:17.702761 4295 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 20:00:17.703159 4295 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 20:00:17.705626 4295 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-2/data/instance:
uuid: "7f43a6a14ef547f2aeb22ce2b334383e"
format_stamp: "Formatted at 2026-09-25 20:00:17 on dist-test-slave-9gwk"
I20260925 20:00:17.706432 4295 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-2/wal/instance:
uuid: "7f43a6a14ef547f2aeb22ce2b334383e"
format_stamp: "Formatted at 2026-09-25 20:00:17 on dist-test-slave-9gwk"
I20260925 20:00:17.712131 4295 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 20:00:17.716156 4312 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:17.717829 4295 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 20:00:17.718034 4295 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-2/data,/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-2/wal
uuid: "7f43a6a14ef547f2aeb22ce2b334383e"
format_stamp: "Formatted at 2026-09-25 20:00:17 on dist-test-slave-9gwk"
I20260925 20:00:17.718276 4295 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 20:00:17.748697 4295 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 20:00:17.749692 4295 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 20:00:17.749982 4295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 20:00:17.756373 4224 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } attempt: 3
W20260925 20:00:17.758723 4224 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.3.199.252:37957: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:17.771910 4295 rpc_server.cc:307] RPC server started. Bound to: 127.3.199.252:37957
I20260925 20:00:17.771931 4363 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.199.252:37957 every 8 connection(s)
I20260925 20:00:17.773658 4295 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-2/data/info.pb
I20260925 20:00:17.778146 4364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:17.782922 3871 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu as pid 4295
I20260925 20:00:17.783115 3871 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-2/wal/instance
I20260925 20:00:17.784227 4364 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } has no permanent_uuid. Determining permanent_uuid...
I20260925 20:00:17.799589 4364 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } has no permanent_uuid. Determining permanent_uuid...
I20260925 20:00:17.806818 4364 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } has no permanent_uuid. Determining permanent_uuid...
I20260925 20:00:17.822817 4364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e: Bootstrap starting.
I20260925 20:00:17.826294 4364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:17.827677 4364 log.cc:826] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e: Log is configured to *not* fsync() on all Append() calls
I20260925 20:00:17.830472 4364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e: No bootstrap required, opened a new log
I20260925 20:00:17.835846 4364 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:17.836268 4364 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:17.836477 4364 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f43a6a14ef547f2aeb22ce2b334383e, State: Initialized, Role: FOLLOWER
I20260925 20:00:17.837114 4364 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:17.838003 4372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:17.838238 4372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:17.839076 4364 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: configured and running, proceeding with master startup.
W20260925 20:00:17.851787 4386 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 20:00:17.852084 4386 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 20:00:17.848507 4295 master_runner.cc:187] Error getting master registration for 127.3.199.254:41289: OK, instance_id { permanent_uuid: "169b6de20560440b99cc08af549ce477" instance_seqno: 1790366416812483 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 20:00:17.857816 4295 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:00:17.936024 4224 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } attempt: 4
I20260925 20:00:17.948155 4224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Bootstrap starting.
I20260925 20:00:17.958461 4224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:17.959692 4224 log.cc:826] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Log is configured to *not* fsync() on all Append() calls
I20260925 20:00:17.963048 4224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: No bootstrap required, opened a new log
I20260925 20:00:17.968469 4224 raft_consensus.cc:359] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:17.968961 4224 raft_consensus.cc:385] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:17.969165 4224 raft_consensus.cc:740] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 169b6de20560440b99cc08af549ce477, State: Initialized, Role: FOLLOWER
I20260925 20:00:17.970008 4224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:17.971069 4390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:17.971359 4390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:17.972357 4224 sys_catalog.cc:565] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: configured and running, proceeding with master startup.
W20260925 20:00:17.988997 4404 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 20:00:17.989236 4404 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 20:00:17.992139 4294 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } attempt: 5
W20260925 20:00:18.002180 4318 tablet.cc:2433] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 20:00:18.006840 4294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Bootstrap starting.
I20260925 20:00:17.986307 4158 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: OK, instance_id { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" instance_seqno: 1790366417285213 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 20:00:18.007987 4158 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:00:18.012244 4294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:18.013744 4294 log.cc:826] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Log is configured to *not* fsync() on all Append() calls
I20260925 20:00:18.017283 4294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: No bootstrap required, opened a new log
I20260925 20:00:18.026170 4294 raft_consensus.cc:359] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:18.026614 4294 raft_consensus.cc:385] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:18.026774 4294 raft_consensus.cc:740] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da29ee550f82456fa7c5ce7ea916909f, State: Initialized, Role: FOLLOWER
I20260925 20:00:18.027719 4294 consensus_queue.cc:260] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:18.031603 4408 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:18.032047 4408 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:18.033389 4294 sys_catalog.cc:565] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 20:00:18.049047 4423 catalog_manager.cc:818] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260925 20:00:18.049796 4178 tablet.cc:2433] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 20:00:18.066174 4225 master_runner.cc:264] No leader master found from master da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:18.066314 4225 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:00:18.068470 4390 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:18.068758 4390 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:18.070655 4390 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:18.071942 4269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "169b6de20560440b99cc08af549ce477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da29ee550f82456fa7c5ce7ea916909f" is_pre_election: true
I20260925 20:00:18.072537 4269 raft_consensus.cc:2468] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 169b6de20560440b99cc08af549ce477 in term 0.
I20260925 20:00:18.073906 4175 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 169b6de20560440b99cc08af549ce477, da29ee550f82456fa7c5ce7ea916909f; no voters:
I20260925 20:00:18.074770 4339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "169b6de20560440b99cc08af549ce477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" is_pre_election: true
I20260925 20:00:18.075312 4339 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 169b6de20560440b99cc08af549ce477 in term 0.
I20260925 20:00:18.076431 4390 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:18.076591 4390 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:18.076684 4390 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:18.083631 4390 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:18.084635 4390 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 1 election: Requested vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:18.085919 4339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "169b6de20560440b99cc08af549ce477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f43a6a14ef547f2aeb22ce2b334383e"
I20260925 20:00:18.086081 4269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "169b6de20560440b99cc08af549ce477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da29ee550f82456fa7c5ce7ea916909f"
I20260925 20:00:18.086153 4339 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:18.086282 4269 raft_consensus.cc:3060] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:18.089838 4269 raft_consensus.cc:2468] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 169b6de20560440b99cc08af549ce477 in term 1.
I20260925 20:00:18.089852 4339 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 169b6de20560440b99cc08af549ce477 in term 1.
I20260925 20:00:18.090449 4175 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 169b6de20560440b99cc08af549ce477, da29ee550f82456fa7c5ce7ea916909f; no voters:
I20260925 20:00:18.091157 4390 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:18.091673 4390 raft_consensus.cc:697] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 1 LEADER]: Becoming Leader. State: Replica: 169b6de20560440b99cc08af549ce477, State: Running, Role: LEADER
I20260925 20:00:18.092739 4390 consensus_queue.cc:237] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:18.096151 4424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 169b6de20560440b99cc08af549ce477. Latest consensus state: current_term: 1 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:18.096542 4424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:18.098747 4426 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260925 20:00:18.101209 4426 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260925 20:00:18.112326 4339 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 1 FOLLOWER]: Refusing update from remote peer 169b6de20560440b99cc08af549ce477: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 20:00:18.112389 4269 raft_consensus.cc:1275] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 1 FOLLOWER]: Refusing update from remote peer 169b6de20560440b99cc08af549ce477: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 20:00:18.113533 4390 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:18.113948 4424 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:18.140956 4372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 169b6de20560440b99cc08af549ce477. Latest consensus state: current_term: 1 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:18.141309 4372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:18.145166 4427 mvcc.cc:204] Tried to move back new op lower bound from 7333340848577196032 to 7333340848513298432. Current Snapshot: MvccSnapshot[applied={T|T < 7333340848577196032}]
I20260925 20:00:18.148849 4390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:18.149151 4390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:18.151616 4408 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 169b6de20560440b99cc08af549ce477. Latest consensus state: current_term: 1 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:18.151971 4408 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:18.155602 4426 catalog_manager.cc:1396] Generated new cluster ID: 7efb83138ca94380bf2d8f0f2f91c381
I20260925 20:00:18.155752 4426 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260925 20:00:18.156107 4424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:18.156433 4424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:18.156428 4372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:18.156621 4372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:18.157083 4428 mvcc.cc:204] Tried to move back new op lower bound from 7333340848577196032 to 7333340848513298432. Current Snapshot: MvccSnapshot[applied={T|T < 7333340848577196032 or (T in {7333340848577196032})}]
I20260925 20:00:18.161369 4429 mvcc.cc:204] Tried to move back new op lower bound from 7333340848577196032 to 7333340848513298432. Current Snapshot: MvccSnapshot[applied={T|T < 7333340848577196032}]
I20260925 20:00:18.164930 4408 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:18.165192 4408 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:18.214740 4426 catalog_manager.cc:1419] Generated new certificate authority record
I20260925 20:00:18.219233 4426 catalog_manager.cc:1553] Loading token signing keys...
I20260925 20:00:18.250348 4426 catalog_manager.cc:6219] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Generated new TSK 0
I20260925 20:00:18.251953 4426 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260925 20:00:18.254215 4426 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260925 20:00:18.255653 4426 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: -1
I20260925 20:00:18.283888 3871 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
/tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.199.193:0
--local_ip_for_outbound_sockets=127.3.199.193
--webserver_interface=127.3.199.193
--webserver_port=0
--tserver_master_addrs=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--builtin_ntp_servers=127.3.199.212:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20260925 20:00:18.699615 4437 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 20:00:18.699929 4437 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 20:00:18.700098 4437 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 20:00:18.708736 4437 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 20:00:18.708957 4437 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.199.193
I20260925 20:00:18.719017 4437 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.199.212:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.199.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.199.193
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.199.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 25 Sep 2026 19:43:21 UTC on bdcb31816ec0
build id 13997
ASAN enabled
I20260925 20:00:18.720767 4437 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 20:00:18.722707 4437 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 20:00:18.733368 4442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:18.733516 4443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:18.734045 4445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 20:00:18.734411 4437 server_base.cc:1061] running on GCE node
I20260925 20:00:18.735100 4437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 20:00:18.736054 4437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 20:00:18.737289 4437 hybrid_clock.cc:648] HybridClock initialized: now 1790366418737196 us; error 87 us; skew 500 ppm
I20260925 20:00:18.737629 4437 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 20:00:18.739873 4437 webserver.cc:533] Webserver started at http://127.3.199.193:35125/ using document root <none> and password file <none>
I20260925 20:00:18.740870 4437 fs_manager.cc:362] Metadata directory not provided
I20260925 20:00:18.741015 4437 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 20:00:18.741392 4437 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 20:00:18.744040 4437 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/data/instance:
uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
format_stamp: "Formatted at 2026-09-25 20:00:18 on dist-test-slave-9gwk"
I20260925 20:00:18.744910 4437 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/instance:
uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
format_stamp: "Formatted at 2026-09-25 20:00:18 on dist-test-slave-9gwk"
I20260925 20:00:18.750460 4437 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260925 20:00:18.750794 4451 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 169b6de20560440b99cc08af549ce477)
I20260925 20:00:18.750996 4451 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:18.752588 4451 leader_election.cc:290] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897)
I20260925 20:00:18.753213 4268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da29ee550f82456fa7c5ce7ea916909f" is_pre_election: true
I20260925 20:00:18.753244 4199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "169b6de20560440b99cc08af549ce477" is_pre_election: true
I20260925 20:00:18.754106 4315 leader_election.cc:304] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f43a6a14ef547f2aeb22ce2b334383e; no voters: 169b6de20560440b99cc08af549ce477, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:18.755123 4451 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:00:18.756649 4453 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:18.757488 4452 raft_consensus.cc:493] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 169b6de20560440b99cc08af549ce477)
I20260925 20:00:18.757705 4452 raft_consensus.cc:515] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:18.758584 4437 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260925 20:00:18.758795 4437 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/data,/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal
uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
format_stamp: "Formatted at 2026-09-25 20:00:18 on dist-test-slave-9gwk"
I20260925 20:00:18.759269 4452 leader_election.cc:290] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:18.759107 4437 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 20:00:18.759825 4338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da29ee550f82456fa7c5ce7ea916909f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" is_pre_election: true
I20260925 20:00:18.760073 4338 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da29ee550f82456fa7c5ce7ea916909f in term 1.
I20260925 20:00:18.760519 4199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da29ee550f82456fa7c5ce7ea916909f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "169b6de20560440b99cc08af549ce477" is_pre_election: true
I20260925 20:00:18.760741 4246 leader_election.cc:304] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f; no voters:
I20260925 20:00:18.761178 4452 raft_consensus.cc:2804] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 20:00:18.761324 4452 raft_consensus.cc:493] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 169b6de20560440b99cc08af549ce477)
I20260925 20:00:18.761431 4452 raft_consensus.cc:3060] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 1 FOLLOWER]: Advancing to term 2
I20260925 20:00:18.763490 4452 raft_consensus.cc:515] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:18.764179 4452 leader_election.cc:290] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 2 election: Requested vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:18.764684 4199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da29ee550f82456fa7c5ce7ea916909f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "169b6de20560440b99cc08af549ce477"
I20260925 20:00:18.764791 4338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da29ee550f82456fa7c5ce7ea916909f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7f43a6a14ef547f2aeb22ce2b334383e"
I20260925 20:00:18.764976 4338 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 1 FOLLOWER]: Advancing to term 2
I20260925 20:00:18.767159 4338 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da29ee550f82456fa7c5ce7ea916909f in term 2.
I20260925 20:00:18.767673 4246 leader_election.cc:304] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f; no voters: 169b6de20560440b99cc08af549ce477
I20260925 20:00:18.768005 4452 raft_consensus.cc:2804] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 2 FOLLOWER]: Leader election won for term 2
I20260925 20:00:18.768429 4452 raft_consensus.cc:697] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 2 LEADER]: Becoming Leader. State: Replica: da29ee550f82456fa7c5ce7ea916909f, State: Running, Role: LEADER
I20260925 20:00:18.768872 4452 consensus_queue.cc:237] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:18.770277 4458 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: New leader da29ee550f82456fa7c5ce7ea916909f. Latest consensus state: current_term: 2 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:18.770470 4458 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:18.771044 4459 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260925 20:00:18.776259 4459 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260925 20:00:18.777670 4459 catalog_manager.cc:1308] Loaded cluster ID: 7efb83138ca94380bf2d8f0f2f91c381
I20260925 20:00:18.777822 4459 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260925 20:00:18.781678 4459 catalog_manager.cc:1553] Loading token signing keys...
I20260925 20:00:18.783263 4459 catalog_manager.cc:6230] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Loaded TSK: 0
I20260925 20:00:18.784915 4459 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260925 20:00:18.786044 4459 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260925 20:00:18.787098 4459 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: -1
I20260925 20:00:18.793783 4437 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 20:00:18.795254 4437 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 20:00:18.795717 4437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 20:00:18.801301 4437 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260925 20:00:18.804425 4437 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 20:00:18.804587 4437 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:18.804759 4437 ts_tablet_manager.cc:616] Registered 0 tablets
I20260925 20:00:18.804854 4437 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:18.807108 4338 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 2 FOLLOWER]: Rejecting Update request from peer 169b6de20560440b99cc08af549ce477 for earlier term 1. Current term is 2. Ops: []
I20260925 20:00:18.807971 4461 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260925 20:00:18.808706 4465 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 1 LEADER]: Stepping down as leader of term 1
I20260925 20:00:18.808838 4465 raft_consensus.cc:740] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 169b6de20560440b99cc08af549ce477, State: Running, Role: LEADER
I20260925 20:00:18.809288 4465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:18.809881 4465 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 1 FOLLOWER]: Advancing to term 2
I20260925 20:00:18.855016 4386 catalog_manager.cc:1308] Loaded cluster ID: 7efb83138ca94380bf2d8f0f2f91c381
I20260925 20:00:18.855135 4386 catalog_manager.cc:1601] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e: loading cluster ID for follower catalog manager: success
I20260925 20:00:18.859408 4386 catalog_manager.cc:1623] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e: acquiring CA information for follower catalog manager: success
I20260925 20:00:18.863569 4386 catalog_manager.cc:1651] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 20:00:18.869323 4437 rpc_server.cc:307] RPC server started. Bound to: 127.3.199.193:34741
I20260925 20:00:18.869361 4570 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.199.193:34741 every 8 connection(s)
I20260925 20:00:18.871380 4437 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/data/info.pb
I20260925 20:00:18.879679 3871 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu as pid 4437
I20260925 20:00:18.879971 3871 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/instance
I20260925 20:00:18.887845 3871 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
/tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.199.194:0
--local_ip_for_outbound_sockets=127.3.199.194
--webserver_interface=127.3.199.194
--webserver_port=0
--tserver_master_addrs=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--builtin_ntp_servers=127.3.199.212:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260925 20:00:18.926828 4571 heartbeater.cc:344] Connected to a master server at 127.3.199.252:37957
I20260925 20:00:18.927254 4571 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:18.928160 4571 heartbeater.cc:507] Master 127.3.199.252:37957 requested a full tablet report, sending...
I20260925 20:00:18.929064 4572 heartbeater.cc:344] Connected to a master server at 127.3.199.254:41289
I20260925 20:00:18.929190 4572 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:18.929700 4572 heartbeater.cc:507] Master 127.3.199.254:41289 requested a full tablet report, sending...
I20260925 20:00:18.931295 4329 ts_manager.cc:194] Registered new tserver with Master: 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:18.933421 4189 ts_manager.cc:194] Registered new tserver with Master: 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:18.937052 4573 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:18.937225 4573 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:18.937680 4573 heartbeater.cc:507] Master 127.3.199.253:41897 requested a full tablet report, sending...
I20260925 20:00:18.939230 4259 ts_manager.cc:194] Registered new tserver with Master: 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:18.941097 4259 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.3.199.193:41285
I20260925 20:00:18.992192 4404 catalog_manager.cc:1651] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 20:00:19.221257 4199 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 2 FOLLOWER]: Refusing update from remote peer da29ee550f82456fa7c5ce7ea916909f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260925 20:00:19.222381 4458 consensus_queue.cc:1048] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [LEADER]: Connected to new peer: Peer: permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260925 20:00:19.224884 4465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da29ee550f82456fa7c5ce7ea916909f. Latest consensus state: current_term: 2 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:19.225144 4465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:19.228269 4452 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:19.228675 4452 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:19.229863 4338 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 2 FOLLOWER]: Refusing update from remote peer da29ee550f82456fa7c5ce7ea916909f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260925 20:00:19.230878 4458 consensus_queue.cc:1048] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260925 20:00:19.242290 4465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:19.242569 4465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:19.244742 4451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: SysCatalogTable state changed. Reason: New leader da29ee550f82456fa7c5ce7ea916909f. Latest consensus state: current_term: 2 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:19.244674 4585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:19.244987 4451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:19.245029 4585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: This master's current role is: FOLLOWER
W20260925 20:00:19.376731 4580 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 20:00:19.377089 4580 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 20:00:19.377249 4580 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 20:00:19.386734 4580 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 20:00:19.387010 4580 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.199.194
I20260925 20:00:19.398049 4580 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.199.212:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.199.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.199.194
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.199.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 25 Sep 2026 19:43:21 UTC on bdcb31816ec0
build id 13997
ASAN enabled
I20260925 20:00:19.400002 4580 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 20:00:19.402081 4580 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 20:00:19.414436 4590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:19.415668 4591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 20:00:19.425376 4580 server_base.cc:1061] running on GCE node
W20260925 20:00:19.425516 4593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 20:00:19.426235 4580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 20:00:19.427546 4580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 20:00:19.428831 4580 hybrid_clock.cc:648] HybridClock initialized: now 1790366419428729 us; error 98 us; skew 500 ppm
I20260925 20:00:19.429342 4580 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 20:00:19.432116 4580 webserver.cc:533] Webserver started at http://127.3.199.194:39209/ using document root <none> and password file <none>
I20260925 20:00:19.433113 4580 fs_manager.cc:362] Metadata directory not provided
I20260925 20:00:19.433285 4580 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 20:00:19.433764 4580 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 20:00:19.437322 4580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/data/instance:
uuid: "46a6f571dfa749b7905d8458d4c929b0"
format_stamp: "Formatted at 2026-09-25 20:00:19 on dist-test-slave-9gwk"
I20260925 20:00:19.438249 4580 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/instance:
uuid: "46a6f571dfa749b7905d8458d4c929b0"
format_stamp: "Formatted at 2026-09-25 20:00:19 on dist-test-slave-9gwk"
I20260925 20:00:19.443818 4573 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:19.446054 4580 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260925 20:00:19.450109 4599 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:19.451728 4580 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 20:00:19.451962 4580 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/data,/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal
uuid: "46a6f571dfa749b7905d8458d4c929b0"
format_stamp: "Formatted at 2026-09-25 20:00:19 on dist-test-slave-9gwk"
I20260925 20:00:19.452234 4580 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 20:00:19.478577 4580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 20:00:19.479724 4580 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 20:00:19.480093 4580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 20:00:19.481333 4580 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260925 20:00:19.483343 4580 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 20:00:19.483498 4580 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:19.483776 4580 ts_tablet_manager.cc:616] Registered 0 tablets
I20260925 20:00:19.483886 4580 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:19.538291 4580 rpc_server.cc:307] RPC server started. Bound to: 127.3.199.194:36647
I20260925 20:00:19.538389 4711 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.199.194:36647 every 8 connection(s)
I20260925 20:00:19.539870 4580 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/data/info.pb
I20260925 20:00:19.543300 3871 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu as pid 4580
I20260925 20:00:19.543515 3871 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/instance
I20260925 20:00:19.549893 3871 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
/tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.199.195:0
--local_ip_for_outbound_sockets=127.3.199.195
--webserver_interface=127.3.199.195
--webserver_port=0
--tserver_master_addrs=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--builtin_ntp_servers=127.3.199.212:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260925 20:00:19.564569 4712 heartbeater.cc:344] Connected to a master server at 127.3.199.252:37957
I20260925 20:00:19.564690 4714 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:19.564800 4713 heartbeater.cc:344] Connected to a master server at 127.3.199.254:41289
I20260925 20:00:19.565038 4713 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:19.565033 4712 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:19.565033 4714 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:19.566263 4714 heartbeater.cc:507] Master 127.3.199.253:41897 requested a full tablet report, sending...
I20260925 20:00:19.566527 4712 heartbeater.cc:507] Master 127.3.199.252:37957 requested a full tablet report, sending...
I20260925 20:00:19.567154 4713 heartbeater.cc:507] Master 127.3.199.254:41289 requested a full tablet report, sending...
I20260925 20:00:19.568022 4329 ts_manager.cc:194] Registered new tserver with Master: 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:19.569113 4259 ts_manager.cc:194] Registered new tserver with Master: 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:19.569591 4189 ts_manager.cc:194] Registered new tserver with Master: 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:19.570616 4259 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.3.199.194:46619
I20260925 20:00:19.649482 4458 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:19.649771 4458 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:19.728616 4465 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:19.728880 4465 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:19.729676 4465 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:19.730278 4268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "169b6de20560440b99cc08af549ce477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da29ee550f82456fa7c5ce7ea916909f" is_pre_election: true
I20260925 20:00:19.730271 4338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "169b6de20560440b99cc08af549ce477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" is_pre_election: true
I20260925 20:00:19.731073 4176 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:19.731907 4465 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260925 20:00:19.936975 4722 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 20:00:19.937469 4722 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 20:00:19.937637 4722 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 20:00:19.946511 4722 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 20:00:19.946826 4722 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.199.195
I20260925 20:00:19.957471 4722 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.199.212:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.199.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.199.195
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.199.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 25 Sep 2026 19:43:21 UTC on bdcb31816ec0
build id 13997
ASAN enabled
I20260925 20:00:19.959187 4722 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 20:00:19.961035 4722 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 20:00:19.971649 4729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:19.971964 4728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:19.971619 4731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 20:00:19.972960 4722 server_base.cc:1061] running on GCE node
I20260925 20:00:19.973606 4722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 20:00:19.974644 4722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 20:00:19.975869 4722 hybrid_clock.cc:648] HybridClock initialized: now 1790366419975797 us; error 64 us; skew 500 ppm
I20260925 20:00:19.976261 4722 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 20:00:19.978955 4722 webserver.cc:533] Webserver started at http://127.3.199.195:39145/ using document root <none> and password file <none>
I20260925 20:00:19.979880 4722 fs_manager.cc:362] Metadata directory not provided
I20260925 20:00:19.980000 4722 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 20:00:19.980577 4722 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 20:00:19.984166 4722 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/data/instance:
uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
format_stamp: "Formatted at 2026-09-25 20:00:19 on dist-test-slave-9gwk"
I20260925 20:00:19.985281 4722 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/instance:
uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
format_stamp: "Formatted at 2026-09-25 20:00:19 on dist-test-slave-9gwk"
I20260925 20:00:19.991797 4722 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260925 20:00:19.995729 4737 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:19.997852 4722 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 20:00:19.998214 4722 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/data,/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal
uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
format_stamp: "Formatted at 2026-09-25 20:00:19 on dist-test-slave-9gwk"
I20260925 20:00:19.998466 4722 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 20:00:20.025964 4722 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 20:00:20.027032 4722 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 20:00:20.027381 4722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 20:00:20.028654 4722 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260925 20:00:20.030671 4722 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 20:00:20.030782 4722 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:20.030992 4722 ts_tablet_manager.cc:616] Registered 0 tablets
I20260925 20:00:20.031083 4722 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:20.073415 4714 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:20.075906 4722 rpc_server.cc:307] RPC server started. Bound to: 127.3.199.195:34395
I20260925 20:00:20.076040 4849 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.199.195:34395 every 8 connection(s)
I20260925 20:00:20.077535 4722 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/data/info.pb
I20260925 20:00:20.086184 3871 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu as pid 4722
I20260925 20:00:20.086467 3871 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/instance
I20260925 20:00:20.097613 4850 heartbeater.cc:344] Connected to a master server at 127.3.199.252:37957
I20260925 20:00:20.098019 4850 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:20.101485 4851 heartbeater.cc:344] Connected to a master server at 127.3.199.254:41289
I20260925 20:00:20.101671 4851 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:20.102089 4850 heartbeater.cc:507] Master 127.3.199.252:37957 requested a full tablet report, sending...
I20260925 20:00:20.102231 4851 heartbeater.cc:507] Master 127.3.199.254:41289 requested a full tablet report, sending...
I20260925 20:00:20.104164 4189 ts_manager.cc:194] Registered new tserver with Master: dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:20.104453 4329 ts_manager.cc:194] Registered new tserver with Master: dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:20.110234 4852 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:20.110430 4852 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:20.110894 4852 heartbeater.cc:507] Master 127.3.199.253:41897 requested a full tablet report, sending...
I20260925 20:00:20.111799 4259 ts_manager.cc:194] Registered new tserver with Master: dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:20.112682 4259 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.3.199.195:52885
I20260925 20:00:20.125344 3871 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260925 20:00:20.164582 4258 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37508:
name: "default.Table_44dac2dae5044f0d8bc0da4a5aafa446"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:20.165177 4257 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37508:
name: "default.Table_daaabef5febc40faa5610a076e6b8b5c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:20.167346 4255 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37508:
name: "default.Table_7722a249d10f4616ba5b5bb2fc149c18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:20.167611 4253 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37508:
name: "default.Table_4ec42ee954664a658b3aad36206c53aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:20.165474 4256 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37508:
name: "default.Table_5fdc6db4e1ee47848136cec254f09a72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:20.168716 4252 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37508:
name: "default.Table_da698ad34fc34a51ae83565db0a9ff48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:20.169977 4251 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37508:
name: "default.Table_56db49c71dc44c579bb1fa4eebb56bbb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:20.167307 4254 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37508:
name: "default.Table_b6819354070b414c8a47d9f19a011a5c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:20.170042 4250 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37508:
name: "default.Table_1c5a83ea353b453da40b24de92d7e850"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:20.163928 4259 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37508:
name: "default.Table_533f83fef17a455883e73ee1c68b252a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 20:00:20.189673 4253 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4ec42ee954664a658b3aad36206c53aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 20:00:20.192765 4257 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_daaabef5febc40faa5610a076e6b8b5c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 20:00:20.194563 4256 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_5fdc6db4e1ee47848136cec254f09a72 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 20:00:20.195878 4252 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_da698ad34fc34a51ae83565db0a9ff48 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 20:00:20.197001 3871 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu pid 4225
I20260925 20:00:20.198500 4289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 20:00:20.266093 4900 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:20.266304 4900 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:20.267324 4900 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:20.267922 4338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "169b6de20560440b99cc08af549ce477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" is_pre_election: true
I20260925 20:00:20.354878 4268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "169b6de20560440b99cc08af549ce477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da29ee550f82456fa7c5ce7ea916909f" is_pre_election: true
I20260925 20:00:20.357343 4175 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:20.357920 4900 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260925 20:00:20.359004 3871 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu with pid 4225
W20260925 20:00:20.398269 4862 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.199.253:41897)
W20260925 20:00:20.407825 4866 negotiation.cc:336] Failed RPC negotiation. Trace:
0925 20:00:20.402160 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.199.253:41897 (local address 127.0.0.1:37522)
0925 20:00:20.403268 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0925 20:00:20.403292 (+ 24us) client_negotiation.cc:175] Beginning negotiation
0925 20:00:20.403496 (+ 204us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0925 20:00:20.405923 (+ 2427us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":832}
W20260925 20:00:20.452636 4573 heartbeater.cc:646] Failed to heartbeat to 127.3.199.253:41897 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:20.588894 4714 heartbeater.cc:646] Failed to heartbeat to 127.3.199.253:41897 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:20.614907 4852 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:20.628826 4904 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
W20260925 20:00:20.628870 4852 heartbeater.cc:646] Failed to heartbeat to 127.3.199.253:41897 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:20.629058 4904 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:20.630040 4904 leader_election.cc:290] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897)
I20260925 20:00:20.630848 4199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "169b6de20560440b99cc08af549ce477" is_pre_election: true
W20260925 20:00:20.644696 4315 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:20.653394 4315 leader_election.cc:336] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:20.653728 4315 leader_election.cc:304] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f43a6a14ef547f2aeb22ce2b334383e; no voters: 169b6de20560440b99cc08af549ce477, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:20.654068 4904 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260925 20:00:20.994376 4909 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:20.994612 4909 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:20.995386 4909 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:20.995946 4338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "169b6de20560440b99cc08af549ce477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" is_pre_election: true
I20260925 20:00:20.996243 4338 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 169b6de20560440b99cc08af549ce477 in term 2.
I20260925 20:00:20.996845 4176 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 169b6de20560440b99cc08af549ce477, 7f43a6a14ef547f2aeb22ce2b334383e; no voters:
I20260925 20:00:20.997195 4909 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260925 20:00:20.997344 4909 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:20.997462 4909 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 2 FOLLOWER]: Advancing to term 3
W20260925 20:00:20.998061 4175 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:20.999775 4909 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:21.000459 4909 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 3 election: Requested vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:21.001086 4338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "169b6de20560440b99cc08af549ce477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7f43a6a14ef547f2aeb22ce2b334383e"
I20260925 20:00:21.001334 4338 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 2 FOLLOWER]: Advancing to term 3
W20260925 20:00:21.001899 4175 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:21.003643 4338 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 169b6de20560440b99cc08af549ce477 in term 3.
I20260925 20:00:21.004477 4176 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 169b6de20560440b99cc08af549ce477, 7f43a6a14ef547f2aeb22ce2b334383e; no voters: da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:21.004805 4909 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 20:00:21.005203 4909 raft_consensus.cc:697] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 3 LEADER]: Becoming Leader. State: Replica: 169b6de20560440b99cc08af549ce477, State: Running, Role: LEADER
I20260925 20:00:21.005530 4909 consensus_queue.cc:237] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:21.006901 4912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 169b6de20560440b99cc08af549ce477. Latest consensus state: current_term: 3 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:21.007181 4912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:21.007839 4914 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260925 20:00:21.009622 4914 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260925 20:00:21.010526 4914 catalog_manager.cc:1308] Loaded cluster ID: 7efb83138ca94380bf2d8f0f2f91c381
I20260925 20:00:21.010629 4914 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260925 20:00:21.011914 4914 catalog_manager.cc:1553] Loading token signing keys...
I20260925 20:00:21.012735 4914 catalog_manager.cc:6230] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Loaded TSK: 0
I20260925 20:00:21.013394 4914 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260925 20:00:21.014117 4914 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260925 20:00:21.014828 4914 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: -1
I20260925 20:00:21.031438 4189 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_1c5a83ea353b453da40b24de92d7e850"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:21.031560 4187 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_da698ad34fc34a51ae83565db0a9ff48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:21.031890 4185 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_44dac2dae5044f0d8bc0da4a5aafa446"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:21.031541 4188 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_56db49c71dc44c579bb1fa4eebb56bbb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:21.032428 4184 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_533f83fef17a455883e73ee1c68b252a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:21.032606 4186 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_4ec42ee954664a658b3aad36206c53aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:21.033872 4183 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_daaabef5febc40faa5610a076e6b8b5c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:21.034039 4181 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_7722a249d10f4616ba5b5bb2fc149c18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:21.034456 4182 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_5fdc6db4e1ee47848136cec254f09a72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:21.033983 4180 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_b6819354070b414c8a47d9f19a011a5c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:21.043253 4338 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 3 FOLLOWER]: Refusing update from remote peer 169b6de20560440b99cc08af549ce477: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20260925 20:00:21.044087 4912 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260925 20:00:21.044404 4175 consensus_peers.cc:597] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 -> Peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Couldn't send request to peer da29ee550f82456fa7c5ce7ea916909f. Status: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 20:00:21.047688 4904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 169b6de20560440b99cc08af549ce477. Latest consensus state: current_term: 3 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:21.047962 4904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:21.049751 4912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:21.050036 4912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:21.050875 4904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:21.051110 4904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: This master's current role is: FOLLOWER
W20260925 20:00:21.052213 4187 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_da698ad34fc34a51ae83565db0a9ff48 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 20:00:21.052244 4188 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_56db49c71dc44c579bb1fa4eebb56bbb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 20:00:21.052516 4185 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_44dac2dae5044f0d8bc0da4a5aafa446 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 20:00:21.052963 4189 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1c5a83ea353b453da40b24de92d7e850 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 20:00:21.087862 4187 catalog_manager.cc:2213] created HMS catalog entry for table [id=c3a1c6d72e6b46c4b253c1c356b17bd7]
I20260925 20:00:21.104775 4187 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_da698ad34fc34a51ae83565db0a9ff48" } new_table_name: "default.Table_c860d14624744708bf4ff50a84517b59" modify_external_catalogs: true
I20260925 20:00:21.126818 4785 tablet_service.cc:1511] Processing CreateTablet for tablet 5f346fa6dc6d46a185f35fec1bab9105 (DEFAULT_TABLE table=default.table_da698ad34fc34a51ae83565db0a9ff48 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.128347 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f346fa6dc6d46a185f35fec1bab9105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.155617 4929 tablet_bootstrap.cc:492] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:21.159704 4929 tablet_bootstrap.cc:654] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.161239 4929 log.cc:826] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e: Log is configured to *not* fsync() on all Append() calls
I20260925 20:00:21.166183 4185 catalog_manager.cc:2213] created HMS catalog entry for table [id=78f973c6a62948e18d7f86d475b64812]
I20260925 20:00:21.166396 4506 tablet_service.cc:1511] Processing CreateTablet for tablet 5f346fa6dc6d46a185f35fec1bab9105 (DEFAULT_TABLE table=default.table_da698ad34fc34a51ae83565db0a9ff48 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.168167 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f346fa6dc6d46a185f35fec1bab9105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.169530 4929 tablet_bootstrap.cc:492] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:21.169860 4929 ts_tablet_manager.cc:1403] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260925 20:00:21.173604 4647 tablet_service.cc:1511] Processing CreateTablet for tablet 5f346fa6dc6d46a185f35fec1bab9105 (DEFAULT_TABLE table=default.table_da698ad34fc34a51ae83565db0a9ff48 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.175221 4647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f346fa6dc6d46a185f35fec1bab9105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.177183 4929 raft_consensus.cc:359] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:21.177628 4929 raft_consensus.cc:385] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.177734 4929 raft_consensus.cc:740] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.178507 4929 consensus_queue.cc:260] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:21.183033 4185 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_44dac2dae5044f0d8bc0da4a5aafa446" } new_table_name: "default.Table_e72a162b7a554904a9ca0cceb5e6b273" modify_external_catalogs: true
W20260925 20:00:21.184875 4184 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_533f83fef17a455883e73ee1c68b252a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 20:00:21.186431 4186 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4ec42ee954664a658b3aad36206c53aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 20:00:21.187721 4183 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_daaabef5febc40faa5610a076e6b8b5c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 20:00:21.189071 4180 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_b6819354070b414c8a47d9f19a011a5c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 20:00:21.190294 4182 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_5fdc6db4e1ee47848136cec254f09a72 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 20:00:21.191556 4181 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_7722a249d10f4616ba5b5bb2fc149c18 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 20:00:21.193883 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 2a8e9b9011114ee4b7b364870ef864b4 (DEFAULT_TABLE table=default.table_44dac2dae5044f0d8bc0da4a5aafa446 [id=78f973c6a62948e18d7f86d475b64812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.194659 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a8e9b9011114ee4b7b364870ef864b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.197973 4646 tablet_service.cc:1511] Processing CreateTablet for tablet 2a8e9b9011114ee4b7b364870ef864b4 (DEFAULT_TABLE table=default.table_44dac2dae5044f0d8bc0da4a5aafa446 [id=78f973c6a62948e18d7f86d475b64812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.198665 4646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a8e9b9011114ee4b7b364870ef864b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.217403 4785 tablet_service.cc:1511] Processing CreateTablet for tablet 2a8e9b9011114ee4b7b364870ef864b4 (DEFAULT_TABLE table=default.table_44dac2dae5044f0d8bc0da4a5aafa446 [id=78f973c6a62948e18d7f86d475b64812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.218066 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a8e9b9011114ee4b7b364870ef864b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.229245 4932 tablet_bootstrap.cc:492] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:21.253235 4932 tablet_bootstrap.cc:654] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.255178 4932 log.cc:826] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482: Log is configured to *not* fsync() on all Append() calls
I20260925 20:00:21.256762 4933 tablet_bootstrap.cc:492] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:21.272408 4851 heartbeater.cc:499] Master 127.3.199.254:41289 was elected leader, sending a full tablet report...
I20260925 20:00:21.273387 4929 ts_tablet_manager.cc:1434] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.103s user 0.039s sys 0.062s
I20260925 20:00:21.276186 4933 tablet_bootstrap.cc:654] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.276480 4189 catalog_manager.cc:2213] created HMS catalog entry for table [id=90221d8ab3b94046bbdff933844c7f22]
I20260925 20:00:21.277745 4929 tablet_bootstrap.cc:492] T 2a8e9b9011114ee4b7b364870ef864b4 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:21.279763 4929 tablet_bootstrap.cc:654] T 2a8e9b9011114ee4b7b364870ef864b4 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.290339 4933 log.cc:826] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0: Log is configured to *not* fsync() on all Append() calls
I20260925 20:00:21.295955 3871 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
/tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.253:41897
--webserver_interface=127.3.199.253
--webserver_port=41541
--builtin_ntp_servers=127.3.199.212:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 20:00:21.304981 4785 tablet_service.cc:1511] Processing CreateTablet for tablet f702fbc80001461298cd1f81e578a05f (DEFAULT_TABLE table=default.table_1c5a83ea353b453da40b24de92d7e850 [id=90221d8ab3b94046bbdff933844c7f22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.305490 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f702fbc80001461298cd1f81e578a05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.306838 4932 tablet_bootstrap.cc:492] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:21.307262 4932 ts_tablet_manager.cc:1403] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.078s user 0.010s sys 0.003s
I20260925 20:00:21.309065 4505 tablet_service.cc:1511] Processing CreateTablet for tablet f702fbc80001461298cd1f81e578a05f (DEFAULT_TABLE table=default.table_1c5a83ea353b453da40b24de92d7e850 [id=90221d8ab3b94046bbdff933844c7f22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.309546 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f702fbc80001461298cd1f81e578a05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.318185 4647 tablet_service.cc:1511] Processing CreateTablet for tablet f702fbc80001461298cd1f81e578a05f (DEFAULT_TABLE table=default.table_1c5a83ea353b453da40b24de92d7e850 [id=90221d8ab3b94046bbdff933844c7f22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.318856 4647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f702fbc80001461298cd1f81e578a05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.325507 4188 catalog_manager.cc:2213] created HMS catalog entry for table [id=86f1c8a16cc14ef9b72c95060313b3ad]
I20260925 20:00:21.325903 4933 tablet_bootstrap.cc:492] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:21.326314 4933 ts_tablet_manager.cc:1403] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.070s user 0.009s sys 0.002s
I20260925 20:00:21.330978 4189 master_service.cc:960] ReplaceTablet: received request to replace tablet 5f346fa6dc6d46a185f35fec1bab9105 from {username='slave'} at 127.0.0.1:48108
W20260925 20:00:21.333297 4853 tablet.cc:2433] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 20:00:21.334239 4932 raft_consensus.cc:359] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:21.334719 4188 master_service.cc:960] ReplaceTablet: received request to replace tablet 5f346fa6dc6d46a185f35fec1bab9105 from {username='slave'} at 127.0.0.1:48108
I20260925 20:00:21.349263 4189 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Sending DeleteTablet for 3 replicas of tablet 5f346fa6dc6d46a185f35fec1bab9105
I20260925 20:00:21.349653 4932 raft_consensus.cc:385] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.349893 4932 raft_consensus.cc:740] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.350196 4189 catalog_manager.cc:6934] ReplaceTablet: tablet 5f346fa6dc6d46a185f35fec1bab9105 deleted and replaced by tablet 983d375d3fda4e63ab93b6cde278e187
I20260925 20:00:21.350554 4189 master_service.cc:960] ReplaceTablet: received request to replace tablet 2a8e9b9011114ee4b7b364870ef864b4 from {username='slave'} at 127.0.0.1:48108
I20260925 20:00:21.351080 4785 tablet_service.cc:1558] Processing DeleteTablet for tablet 5f346fa6dc6d46a185f35fec1bab9105 with delete_type TABLET_DATA_DELETED (replaced by tablet 983d375d3fda4e63ab93b6cde278e187) from {username='slave'} at 127.0.0.1:54514
I20260925 20:00:21.351454 4647 tablet_service.cc:1558] Processing DeleteTablet for tablet 5f346fa6dc6d46a185f35fec1bab9105 with delete_type TABLET_DATA_DELETED (replaced by tablet 983d375d3fda4e63ab93b6cde278e187) from {username='slave'} at 127.0.0.1:57378
I20260925 20:00:21.351784 4932 consensus_queue.cc:260] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:21.353515 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet 5f346fa6dc6d46a185f35fec1bab9105 with delete_type TABLET_DATA_DELETED (replaced by tablet 983d375d3fda4e63ab93b6cde278e187) from {username='slave'} at 127.0.0.1:54948
I20260925 20:00:21.361123 4933 raft_consensus.cc:359] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.366155 4933 raft_consensus.cc:385] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.367197 4188 master_service.cc:960] ReplaceTablet: received request to replace tablet 5f346fa6dc6d46a185f35fec1bab9105 from {username='slave'} at 127.0.0.1:48108
I20260925 20:00:21.367762 4188 master_service.cc:960] ReplaceTablet: received request to replace tablet 2a8e9b9011114ee4b7b364870ef864b4 from {username='slave'} at 127.0.0.1:48108
I20260925 20:00:21.368420 4933 raft_consensus.cc:740] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.369304 4646 tablet_service.cc:1511] Processing CreateTablet for tablet ee843d55757347ef9942dffd917c0ff8 (DEFAULT_TABLE table=default.table_56db49c71dc44c579bb1fa4eebb56bbb [id=86f1c8a16cc14ef9b72c95060313b3ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.369786 4646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee843d55757347ef9942dffd917c0ff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.371009 4938 tablet_replica.cc:333] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:21.371596 4938 raft_consensus.cc:2243] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:21.372071 4938 raft_consensus.cc:2272] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:21.372187 4785 tablet_service.cc:1511] Processing CreateTablet for tablet ee843d55757347ef9942dffd917c0ff8 (DEFAULT_TABLE table=default.table_56db49c71dc44c579bb1fa4eebb56bbb [id=86f1c8a16cc14ef9b72c95060313b3ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.372725 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee843d55757347ef9942dffd917c0ff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.374238 4505 tablet_service.cc:1511] Processing CreateTablet for tablet ee843d55757347ef9942dffd917c0ff8 (DEFAULT_TABLE table=default.table_56db49c71dc44c579bb1fa4eebb56bbb [id=86f1c8a16cc14ef9b72c95060313b3ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.374851 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee843d55757347ef9942dffd917c0ff8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 20:00:21.375264 4176 catalog_manager.cc:5043] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): delete failed for tablet 5f346fa6dc6d46a185f35fec1bab9105 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5f346fa6dc6d46a185f35fec1bab9105 already in progress: creating tablet
W20260925 20:00:21.376953 4174 catalog_manager.cc:5043] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): delete failed for tablet 5f346fa6dc6d46a185f35fec1bab9105 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5f346fa6dc6d46a185f35fec1bab9105 already in progress: creating tablet
I20260925 20:00:21.390615 4933 consensus_queue.cc:260] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.393371 4932 ts_tablet_manager.cc:1434] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.086s user 0.013s sys 0.001s
I20260925 20:00:21.395057 4932 tablet_bootstrap.cc:492] T 2a8e9b9011114ee4b7b364870ef864b4 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:21.398413 4932 tablet_bootstrap.cc:654] T 2a8e9b9011114ee4b7b364870ef864b4 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
W20260925 20:00:21.400482 4574 tablet.cc:2433] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 20:00:21.409642 4938 ts_tablet_manager.cc:1916] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:21.411535 4646 tablet_service.cc:1511] Processing CreateTablet for tablet 983d375d3fda4e63ab93b6cde278e187 (DEFAULT_TABLE table=default.table_da698ad34fc34a51ae83565db0a9ff48 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.412075 4646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 983d375d3fda4e63ab93b6cde278e187. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.413928 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 983d375d3fda4e63ab93b6cde278e187 (DEFAULT_TABLE table=default.table_da698ad34fc34a51ae83565db0a9ff48 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.414438 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 983d375d3fda4e63ab93b6cde278e187. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.414451 4784 tablet_service.cc:1511] Processing CreateTablet for tablet 983d375d3fda4e63ab93b6cde278e187 (DEFAULT_TABLE table=default.table_da698ad34fc34a51ae83565db0a9ff48 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.415148 4784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 983d375d3fda4e63ab93b6cde278e187. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.415335 4932 tablet_bootstrap.cc:492] T 2a8e9b9011114ee4b7b364870ef864b4 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:21.415498 4932 ts_tablet_manager.cc:1403] T 2a8e9b9011114ee4b7b364870ef864b4 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.000s
I20260925 20:00:21.417611 4932 raft_consensus.cc:359] T 2a8e9b9011114ee4b7b364870ef864b4 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.417824 4932 raft_consensus.cc:385] T 2a8e9b9011114ee4b7b364870ef864b4 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.417928 4932 raft_consensus.cc:740] T 2a8e9b9011114ee4b7b364870ef864b4 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.427631 4932 consensus_queue.cc:260] T 2a8e9b9011114ee4b7b364870ef864b4 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.433575 4189 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_1c5a83ea353b453da40b24de92d7e850" } new_table_name: "default.Table_b74d63554f344195800ce065c2a3cddf" modify_external_catalogs: true
I20260925 20:00:21.433866 4932 ts_tablet_manager.cc:1434] T 2a8e9b9011114ee4b7b364870ef864b4 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.018s user 0.003s sys 0.000s
I20260925 20:00:21.434088 4572 heartbeater.cc:499] Master 127.3.199.254:41289 was elected leader, sending a full tablet report...
I20260925 20:00:21.434088 4188 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Sending DeleteTablet for 3 replicas of tablet 2a8e9b9011114ee4b7b364870ef864b4
I20260925 20:00:21.434605 4188 catalog_manager.cc:6934] ReplaceTablet: tablet 2a8e9b9011114ee4b7b364870ef864b4 deleted and replaced by tablet d96619f6f7e04b789ba8d09f334ff105
I20260925 20:00:21.434979 4188 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_30a6c0d501b04bfc882cec2f43c06f83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:21.437108 4932 tablet_bootstrap.cc:492] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:21.438773 4647 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a8e9b9011114ee4b7b364870ef864b4 with delete_type TABLET_DATA_DELETED (replaced by tablet d96619f6f7e04b789ba8d09f334ff105) from {username='slave'} at 127.0.0.1:57378
I20260925 20:00:21.439625 4932 tablet_bootstrap.cc:654] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.439958 4783 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a8e9b9011114ee4b7b364870ef864b4 with delete_type TABLET_DATA_DELETED (replaced by tablet d96619f6f7e04b789ba8d09f334ff105) from {username='slave'} at 127.0.0.1:54514
I20260925 20:00:21.440311 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet 5f346fa6dc6d46a185f35fec1bab9105 with delete_type TABLET_DATA_DELETED (replaced by tablet 983d375d3fda4e63ab93b6cde278e187) from {username='slave'} at 127.0.0.1:54948
I20260925 20:00:21.440367 4506 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a8e9b9011114ee4b7b364870ef864b4 with delete_type TABLET_DATA_DELETED (replaced by tablet d96619f6f7e04b789ba8d09f334ff105) from {username='slave'} at 127.0.0.1:54948
I20260925 20:00:21.440618 4941 tablet_replica.cc:333] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:21.441116 4941 raft_consensus.cc:2243] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260925 20:00:21.441479 4174 catalog_manager.cc:5043] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): delete failed for tablet 2a8e9b9011114ee4b7b364870ef864b4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2a8e9b9011114ee4b7b364870ef864b4 already in progress: creating tablet
I20260925 20:00:21.441553 4941 raft_consensus.cc:2272] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:21.443526 4941 ts_tablet_manager.cc:1916] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:21.449580 4647 tablet_service.cc:1511] Processing CreateTablet for tablet d96619f6f7e04b789ba8d09f334ff105 (DEFAULT_TABLE table=default.table_44dac2dae5044f0d8bc0da4a5aafa446 [id=78f973c6a62948e18d7f86d475b64812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.450048 4647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d96619f6f7e04b789ba8d09f334ff105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.450352 4506 tablet_service.cc:1511] Processing CreateTablet for tablet d96619f6f7e04b789ba8d09f334ff105 (DEFAULT_TABLE table=default.table_44dac2dae5044f0d8bc0da4a5aafa446 [id=78f973c6a62948e18d7f86d475b64812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.450755 4941 ts_tablet_manager.cc:1929] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 20:00:21.450927 4941 log.cc:1199] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/5f346fa6dc6d46a185f35fec1bab9105
I20260925 20:00:21.450965 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d96619f6f7e04b789ba8d09f334ff105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.451385 4941 ts_tablet_manager.cc:1950] T 5f346fa6dc6d46a185f35fec1bab9105 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:21.453378 4783 tablet_service.cc:1511] Processing CreateTablet for tablet d96619f6f7e04b789ba8d09f334ff105 (DEFAULT_TABLE table=default.table_44dac2dae5044f0d8bc0da4a5aafa446 [id=78f973c6a62948e18d7f86d475b64812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.454118 4783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d96619f6f7e04b789ba8d09f334ff105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.465668 4933 ts_tablet_manager.cc:1434] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.139s user 0.009s sys 0.009s
I20260925 20:00:21.466071 4713 heartbeater.cc:499] Master 127.3.199.254:41289 was elected leader, sending a full tablet report...
I20260925 20:00:21.468540 4933 tablet_bootstrap.cc:492] T 5f346fa6dc6d46a185f35fec1bab9105 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:21.470467 4933 tablet_bootstrap.cc:654] T 5f346fa6dc6d46a185f35fec1bab9105 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.472720 4932 tablet_bootstrap.cc:492] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:21.472925 4932 ts_tablet_manager.cc:1403] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.036s user 0.004s sys 0.000s
I20260925 20:00:21.473721 4932 raft_consensus.cc:359] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.473946 4932 raft_consensus.cc:385] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.474043 4932 raft_consensus.cc:740] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.474426 4932 consensus_queue.cc:260] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.477607 4929 tablet_bootstrap.cc:492] T 2a8e9b9011114ee4b7b364870ef864b4 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:21.477779 4929 ts_tablet_manager.cc:1403] T 2a8e9b9011114ee4b7b364870ef864b4 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.200s user 0.039s sys 0.079s
I20260925 20:00:21.478502 4932 ts_tablet_manager.cc:1434] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260925 20:00:21.478899 4929 raft_consensus.cc:359] T 2a8e9b9011114ee4b7b364870ef864b4 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.479091 4929 raft_consensus.cc:385] T 2a8e9b9011114ee4b7b364870ef864b4 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.479180 4929 raft_consensus.cc:740] T 2a8e9b9011114ee4b7b364870ef864b4 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.479463 4929 consensus_queue.cc:260] T 2a8e9b9011114ee4b7b364870ef864b4 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.479959 4929 ts_tablet_manager.cc:1434] T 2a8e9b9011114ee4b7b364870ef864b4 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260925 20:00:21.480255 4929 tablet_bootstrap.cc:492] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:21.481992 4932 tablet_bootstrap.cc:492] T ee843d55757347ef9942dffd917c0ff8 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:21.483801 4929 tablet_bootstrap.cc:654] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.485646 4176 catalog_manager.cc:5063] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): tablet 5f346fa6dc6d46a185f35fec1bab9105 (table default.table_da698ad34fc34a51ae83565db0a9ff48 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]) successfully deleted
I20260925 20:00:21.486074 4941 tablet_replica.cc:333] T 2a8e9b9011114ee4b7b364870ef864b4 P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:21.486299 4941 raft_consensus.cc:2243] T 2a8e9b9011114ee4b7b364870ef864b4 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:21.486508 4941 raft_consensus.cc:2272] T 2a8e9b9011114ee4b7b364870ef864b4 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:21.487371 4941 ts_tablet_manager.cc:1916] T 2a8e9b9011114ee4b7b364870ef864b4 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:21.489421 4938 ts_tablet_manager.cc:1929] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 20:00:21.489620 4938 log.cc:1199] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/5f346fa6dc6d46a185f35fec1bab9105
I20260925 20:00:21.490125 4938 ts_tablet_manager.cc:1950] T 5f346fa6dc6d46a185f35fec1bab9105 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:21.492272 4176 catalog_manager.cc:5063] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395): tablet 5f346fa6dc6d46a185f35fec1bab9105 (table default.table_da698ad34fc34a51ae83565db0a9ff48 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]) successfully deleted
I20260925 20:00:21.495903 4184 catalog_manager.cc:2213] created HMS catalog entry for table [id=25f6e1ade43c4519bed519037ee8f761]
I20260925 20:00:21.496680 4941 ts_tablet_manager.cc:1929] T 2a8e9b9011114ee4b7b364870ef864b4 P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 20:00:21.496838 4941 log.cc:1199] T 2a8e9b9011114ee4b7b364870ef864b4 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/2a8e9b9011114ee4b7b364870ef864b4
I20260925 20:00:21.497215 4941 ts_tablet_manager.cc:1950] T 2a8e9b9011114ee4b7b364870ef864b4 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:21.497431 4933 tablet_bootstrap.cc:492] T 5f346fa6dc6d46a185f35fec1bab9105 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:21.497591 4933 ts_tablet_manager.cc:1403] T 5f346fa6dc6d46a185f35fec1bab9105 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.002s
I20260925 20:00:21.498622 4176 catalog_manager.cc:5063] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): tablet 2a8e9b9011114ee4b7b364870ef864b4 (table default.table_44dac2dae5044f0d8bc0da4a5aafa446 [id=78f973c6a62948e18d7f86d475b64812]) successfully deleted
I20260925 20:00:21.499029 4933 raft_consensus.cc:359] T 5f346fa6dc6d46a185f35fec1bab9105 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:21.499243 4933 raft_consensus.cc:385] T 5f346fa6dc6d46a185f35fec1bab9105 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.499360 4933 raft_consensus.cc:740] T 5f346fa6dc6d46a185f35fec1bab9105 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.499665 4933 consensus_queue.cc:260] T 5f346fa6dc6d46a185f35fec1bab9105 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:21.503862 4933 ts_tablet_manager.cc:1434] T 5f346fa6dc6d46a185f35fec1bab9105 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260925 20:00:21.504220 4929 tablet_bootstrap.cc:492] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:21.504442 4929 ts_tablet_manager.cc:1403] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.024s user 0.009s sys 0.001s
I20260925 20:00:21.504482 4938 tablet_replica.cc:333] T 2a8e9b9011114ee4b7b364870ef864b4 P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:21.504719 4938 raft_consensus.cc:2243] T 2a8e9b9011114ee4b7b364870ef864b4 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:21.504933 4938 raft_consensus.cc:2272] T 2a8e9b9011114ee4b7b364870ef864b4 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:21.505256 4929 raft_consensus.cc:359] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.505456 4929 raft_consensus.cc:385] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.505537 4929 raft_consensus.cc:740] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.505836 4929 consensus_queue.cc:260] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.506361 4929 ts_tablet_manager.cc:1434] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260925 20:00:21.506793 4929 tablet_bootstrap.cc:492] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:21.508641 4929 tablet_bootstrap.cc:654] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.509941 4933 tablet_bootstrap.cc:492] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:21.511705 4932 tablet_bootstrap.cc:654] T ee843d55757347ef9942dffd917c0ff8 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.515620 4933 tablet_bootstrap.cc:654] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.516530 4506 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a8e9b9011114ee4b7b364870ef864b4 with delete_type TABLET_DATA_DELETED (replaced by tablet d96619f6f7e04b789ba8d09f334ff105) from {username='slave'} at 127.0.0.1:54948
I20260925 20:00:21.516852 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet 5f346fa6dc6d46a185f35fec1bab9105 with delete_type TABLET_DATA_DELETED (replaced by tablet 983d375d3fda4e63ab93b6cde278e187) from {username='slave'} at 127.0.0.1:54948
I20260925 20:00:21.517653 4938 ts_tablet_manager.cc:1916] T 2a8e9b9011114ee4b7b364870ef864b4 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:21.523254 4933 tablet_bootstrap.cc:492] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:21.523491 4933 ts_tablet_manager.cc:1403] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.004s
I20260925 20:00:21.521435 4929 tablet_bootstrap.cc:492] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:21.524108 4929 ts_tablet_manager.cc:1403] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20260925 20:00:21.524998 4933 raft_consensus.cc:359] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.525027 4929 raft_consensus.cc:359] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.525218 4929 raft_consensus.cc:385] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.525238 4933 raft_consensus.cc:385] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.525309 4929 raft_consensus.cc:740] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.525319 4933 raft_consensus.cc:740] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.525555 4929 consensus_queue.cc:260] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.526093 4929 ts_tablet_manager.cc:1434] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 20:00:21.526420 4929 tablet_bootstrap.cc:492] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:21.526696 4646 tablet_service.cc:1511] Processing CreateTablet for tablet 460399668df444c5af242442af590e1a (DEFAULT_TABLE table=default.table_533f83fef17a455883e73ee1c68b252a [id=25f6e1ade43c4519bed519037ee8f761]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.526845 4932 tablet_bootstrap.cc:492] T ee843d55757347ef9942dffd917c0ff8 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:21.527025 4932 ts_tablet_manager.cc:1403] T ee843d55757347ef9942dffd917c0ff8 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.045s user 0.002s sys 0.004s
I20260925 20:00:21.527124 4646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 460399668df444c5af242442af590e1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.527863 4932 raft_consensus.cc:359] T ee843d55757347ef9942dffd917c0ff8 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.528115 4929 tablet_bootstrap.cc:654] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.528162 4932 raft_consensus.cc:385] T ee843d55757347ef9942dffd917c0ff8 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.528245 4932 raft_consensus.cc:740] T ee843d55757347ef9942dffd917c0ff8 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.528543 4932 consensus_queue.cc:260] T ee843d55757347ef9942dffd917c0ff8 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.531715 4932 ts_tablet_manager.cc:1434] T ee843d55757347ef9942dffd917c0ff8 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260925 20:00:21.532079 4932 tablet_bootstrap.cc:492] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:21.533886 4933 consensus_queue.cc:260] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.540608 4785 tablet_service.cc:1511] Processing CreateTablet for tablet 460399668df444c5af242442af590e1a (DEFAULT_TABLE table=default.table_533f83fef17a455883e73ee1c68b252a [id=25f6e1ade43c4519bed519037ee8f761]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.541111 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 460399668df444c5af242442af590e1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.544739 4932 tablet_bootstrap.cc:654] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.544930 4186 catalog_manager.cc:2213] created HMS catalog entry for table [id=c47fb6987d0343418ed55e663b34ab74]
I20260925 20:00:21.545208 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 460399668df444c5af242442af590e1a (DEFAULT_TABLE table=default.table_533f83fef17a455883e73ee1c68b252a [id=25f6e1ade43c4519bed519037ee8f761]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.545742 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 460399668df444c5af242442af590e1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.540429 4933 ts_tablet_manager.cc:1434] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.017s user 0.002s sys 0.001s
I20260925 20:00:21.548089 4933 tablet_bootstrap.cc:492] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:21.549911 4933 tablet_bootstrap.cc:654] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
W20260925 20:00:21.555383 4715 tablet.cc:2433] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 20:00:21.557808 4929 tablet_bootstrap.cc:492] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:21.558032 4929 ts_tablet_manager.cc:1403] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.032s user 0.000s sys 0.003s
I20260925 20:00:21.558976 4929 raft_consensus.cc:359] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.559203 4929 raft_consensus.cc:385] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.559284 4929 raft_consensus.cc:740] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.559602 4929 consensus_queue.cc:260] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.560176 4929 ts_tablet_manager.cc:1434] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260925 20:00:21.560552 4929 tablet_bootstrap.cc:492] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:21.560603 4647 tablet_service.cc:1558] Processing DeleteTablet for tablet 5f346fa6dc6d46a185f35fec1bab9105 with delete_type TABLET_DATA_DELETED (replaced by tablet 983d375d3fda4e63ab93b6cde278e187) from {username='slave'} at 127.0.0.1:57378
I20260925 20:00:21.560946 4939 tablet_replica.cc:333] T 5f346fa6dc6d46a185f35fec1bab9105 P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:21.561533 4939 raft_consensus.cc:2243] T 5f346fa6dc6d46a185f35fec1bab9105 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:21.561968 4939 raft_consensus.cc:2272] T 5f346fa6dc6d46a185f35fec1bab9105 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:21.562336 4929 tablet_bootstrap.cc:654] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.563913 4932 tablet_bootstrap.cc:492] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:21.564105 4932 ts_tablet_manager.cc:1403] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.032s user 0.004s sys 0.000s
I20260925 20:00:21.564634 4939 ts_tablet_manager.cc:1916] T 5f346fa6dc6d46a185f35fec1bab9105 P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:21.564977 4932 raft_consensus.cc:359] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.565274 4932 raft_consensus.cc:385] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.565372 4932 raft_consensus.cc:740] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.567171 4932 consensus_queue.cc:260] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.569864 4938 ts_tablet_manager.cc:1929] T 2a8e9b9011114ee4b7b364870ef864b4 P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 20:00:21.574990 4932 ts_tablet_manager.cc:1434] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.011s user 0.000s sys 0.002s
I20260925 20:00:21.575416 4932 tablet_bootstrap.cc:492] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:21.576143 4938 log.cc:1199] T 2a8e9b9011114ee4b7b364870ef864b4 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/2a8e9b9011114ee4b7b364870ef864b4
I20260925 20:00:21.576930 4938 ts_tablet_manager.cc:1950] T 2a8e9b9011114ee4b7b364870ef864b4 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:21.579134 4176 catalog_manager.cc:5063] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395): tablet 2a8e9b9011114ee4b7b364870ef864b4 (table default.table_44dac2dae5044f0d8bc0da4a5aafa446 [id=78f973c6a62948e18d7f86d475b64812]) successfully deleted
I20260925 20:00:21.589620 4932 tablet_bootstrap.cc:654] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.598031 4186 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_56db49c71dc44c579bb1fa4eebb56bbb" } new_table_name: "default.Table_098b200909f842c88a38f11192f4dc25" modify_external_catalogs: true
I20260925 20:00:21.603615 4785 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a8e9b9011114ee4b7b364870ef864b4 with delete_type TABLET_DATA_DELETED (replaced by tablet d96619f6f7e04b789ba8d09f334ff105) from {username='slave'} at 127.0.0.1:54514
W20260925 20:00:21.605041 4176 catalog_manager.cc:5030] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395): delete failed for tablet 2a8e9b9011114ee4b7b364870ef864b4 because the tablet was not found. No further retry: Not found: Tablet not found: 2a8e9b9011114ee4b7b364870ef864b4
I20260925 20:00:21.606240 4932 tablet_bootstrap.cc:492] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:21.606432 4932 ts_tablet_manager.cc:1403] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.005s
I20260925 20:00:21.607259 4932 raft_consensus.cc:359] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.607620 4932 raft_consensus.cc:385] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.607856 4932 raft_consensus.cc:740] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.608160 4932 consensus_queue.cc:260] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.608721 4932 ts_tablet_manager.cc:1434] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 20:00:21.619228 4647 tablet_service.cc:1511] Processing CreateTablet for tablet dff315b3b8e64ed0838b6241eeb442d3 (DEFAULT_TABLE table=default.table_4ec42ee954664a658b3aad36206c53aa [id=c47fb6987d0343418ed55e663b34ab74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.619793 4647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dff315b3b8e64ed0838b6241eeb442d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.629662 4506 tablet_service.cc:1511] Processing CreateTablet for tablet dff315b3b8e64ed0838b6241eeb442d3 (DEFAULT_TABLE table=default.table_4ec42ee954664a658b3aad36206c53aa [id=c47fb6987d0343418ed55e663b34ab74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.630151 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dff315b3b8e64ed0838b6241eeb442d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.635423 4785 tablet_service.cc:1511] Processing CreateTablet for tablet dff315b3b8e64ed0838b6241eeb442d3 (DEFAULT_TABLE table=default.table_4ec42ee954664a658b3aad36206c53aa [id=c47fb6987d0343418ed55e663b34ab74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:21.635952 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dff315b3b8e64ed0838b6241eeb442d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:21.636236 4932 tablet_bootstrap.cc:492] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:21.638229 4932 tablet_bootstrap.cc:654] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.640506 4646 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a8e9b9011114ee4b7b364870ef864b4 with delete_type TABLET_DATA_DELETED (replaced by tablet d96619f6f7e04b789ba8d09f334ff105) from {username='slave'} at 127.0.0.1:57378
I20260925 20:00:21.640606 4932 tablet_bootstrap.cc:492] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:21.640751 4932 ts_tablet_manager.cc:1403] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260925 20:00:21.641886 4932 raft_consensus.cc:359] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.642115 4932 raft_consensus.cc:385] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.642203 4932 raft_consensus.cc:740] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.642515 4932 consensus_queue.cc:260] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.643213 4932 ts_tablet_manager.cc:1434] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 20:00:21.643545 4932 tablet_bootstrap.cc:492] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:21.645320 4932 tablet_bootstrap.cc:654] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
W20260925 20:00:21.664363 4176 catalog_manager.cc:5030] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): delete failed for tablet 2a8e9b9011114ee4b7b364870ef864b4 because the tablet was not found. No further retry: Not found: Tablet not found: 2a8e9b9011114ee4b7b364870ef864b4
W20260925 20:00:21.664783 4176 catalog_manager.cc:5030] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): delete failed for tablet 5f346fa6dc6d46a185f35fec1bab9105 because the tablet was not found. No further retry: Not found: Tablet not found: 5f346fa6dc6d46a185f35fec1bab9105
I20260925 20:00:21.665866 4939 ts_tablet_manager.cc:1929] T 5f346fa6dc6d46a185f35fec1bab9105 P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 20:00:21.666126 4939 log.cc:1199] T 5f346fa6dc6d46a185f35fec1bab9105 P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/5f346fa6dc6d46a185f35fec1bab9105
I20260925 20:00:21.666796 4939 ts_tablet_manager.cc:1950] T 5f346fa6dc6d46a185f35fec1bab9105 P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:21.669252 4174 catalog_manager.cc:5063] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): tablet 5f346fa6dc6d46a185f35fec1bab9105 (table default.table_da698ad34fc34a51ae83565db0a9ff48 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]) successfully deleted
I20260925 20:00:21.669936 4939 tablet_replica.cc:333] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:21.670234 4939 raft_consensus.cc:2243] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:21.670463 4939 raft_consensus.cc:2272] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:21.671581 4939 ts_tablet_manager.cc:1916] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:21.675393 4933 tablet_bootstrap.cc:492] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:21.675604 4933 ts_tablet_manager.cc:1403] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.128s user 0.016s sys 0.010s
I20260925 20:00:21.676697 4933 raft_consensus.cc:359] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.676944 4933 raft_consensus.cc:385] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.677057 4933 raft_consensus.cc:740] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.677361 4933 consensus_queue.cc:260] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.678094 4933 ts_tablet_manager.cc:1434] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 20:00:21.678501 4933 tablet_bootstrap.cc:492] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:21.682108 4933 tablet_bootstrap.cc:654] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.690665 4184 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_08807903f9884f2a9707d4066e7f2f52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:21.693068 4929 tablet_bootstrap.cc:492] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:21.693434 4929 ts_tablet_manager.cc:1403] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.133s user 0.012s sys 0.039s
I20260925 20:00:21.694120 4932 tablet_bootstrap.cc:492] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:21.694288 4932 ts_tablet_manager.cc:1403] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.051s user 0.016s sys 0.014s
I20260925 20:00:21.694535 4929 raft_consensus.cc:359] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.694831 4929 raft_consensus.cc:385] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.695039 4929 raft_consensus.cc:740] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.695066 4932 raft_consensus.cc:359] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.695269 4932 raft_consensus.cc:385] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.695362 4932 raft_consensus.cc:740] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.695540 4929 consensus_queue.cc:260] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.697053 4932 consensus_queue.cc:260] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.699002 4939 ts_tablet_manager.cc:1929] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 20:00:21.699168 4939 log.cc:1199] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/2a8e9b9011114ee4b7b364870ef864b4
I20260925 20:00:21.699579 4939 ts_tablet_manager.cc:1950] T 2a8e9b9011114ee4b7b364870ef864b4 P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:21.700889 4929 ts_tablet_manager.cc:1434] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260925 20:00:21.702167 4932 ts_tablet_manager.cc:1434] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260925 20:00:21.704478 4929 tablet_bootstrap.cc:492] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:21.705426 4933 tablet_bootstrap.cc:492] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:21.705627 4933 ts_tablet_manager.cc:1403] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.027s user 0.004s sys 0.000s
I20260925 20:00:21.706568 4929 tablet_bootstrap.cc:654] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.706589 4933 raft_consensus.cc:359] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.706857 4933 raft_consensus.cc:385] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.706998 4933 raft_consensus.cc:740] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.707315 4933 consensus_queue.cc:260] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.709381 4933 ts_tablet_manager.cc:1434] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260925 20:00:21.709728 4933 tablet_bootstrap.cc:492] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:21.711983 4933 tablet_bootstrap.cc:654] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.714555 4174 catalog_manager.cc:5063] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): tablet 2a8e9b9011114ee4b7b364870ef864b4 (table default.table_44dac2dae5044f0d8bc0da4a5aafa446 [id=78f973c6a62948e18d7f86d475b64812]) successfully deleted
I20260925 20:00:21.723557 4933 tablet_bootstrap.cc:492] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:21.723841 4933 ts_tablet_manager.cc:1403] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.011s
I20260925 20:00:21.724794 4933 raft_consensus.cc:359] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.725078 4933 raft_consensus.cc:385] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.725219 4933 raft_consensus.cc:740] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.725574 4933 consensus_queue.cc:260] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.728765 4933 ts_tablet_manager.cc:1434] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260925 20:00:21.731720 4933 tablet_bootstrap.cc:492] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:21.734076 4933 tablet_bootstrap.cc:654] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.736162 4929 tablet_bootstrap.cc:492] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:21.736873 4933 tablet_bootstrap.cc:492] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:21.737339 4933 ts_tablet_manager.cc:1403] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260925 20:00:21.737617 4929 ts_tablet_manager.cc:1403] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.033s user 0.005s sys 0.005s
I20260925 20:00:21.739444 4933 raft_consensus.cc:359] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.739663 4933 raft_consensus.cc:385] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.739741 4933 raft_consensus.cc:740] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.739595 4929 raft_consensus.cc:359] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.740054 4929 raft_consensus.cc:385] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.740061 4933 consensus_queue.cc:260] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.740168 4929 raft_consensus.cc:740] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.740684 4933 ts_tablet_manager.cc:1434] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 20:00:21.741879 4933 tablet_bootstrap.cc:492] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:21.744640 4933 tablet_bootstrap.cc:654] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.746081 4929 consensus_queue.cc:260] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.750094 4929 ts_tablet_manager.cc:1434] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.011s user 0.000s sys 0.002s
I20260925 20:00:21.753588 4929 tablet_bootstrap.cc:492] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:21.755434 4929 tablet_bootstrap.cc:654] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:21.795110 4929 tablet_bootstrap.cc:492] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:21.795405 4929 ts_tablet_manager.cc:1403] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.042s user 0.011s sys 0.029s
I20260925 20:00:21.796334 4929 raft_consensus.cc:359] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.796769 4929 raft_consensus.cc:385] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.796932 4929 raft_consensus.cc:740] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.797459 4929 consensus_queue.cc:260] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.801982 4929 ts_tablet_manager.cc:1434] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260925 20:00:21.802801 4933 tablet_bootstrap.cc:492] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:21.803009 4933 ts_tablet_manager.cc:1403] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.062s user 0.018s sys 0.012s
I20260925 20:00:21.803926 4933 raft_consensus.cc:359] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.804141 4933 raft_consensus.cc:385] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:21.804234 4933 raft_consensus.cc:740] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:21.804515 4933 consensus_queue.cc:260] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.805071 4933 ts_tablet_manager.cc:1434] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 20:00:21.868670 4931 raft_consensus.cc:493] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:21.868952 4931 raft_consensus.cc:515] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.870976 4931 leader_election.cc:290] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:21.910004 4931 raft_consensus.cc:493] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:21.910225 4931 raft_consensus.cc:515] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.910873 4931 leader_election.cc:290] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
20:00:21.916 [HMS - ERROR - pool-6-thread-2] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_daaabef5febc40faa5610a076e6b8b5c already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at jdk.proxy2/jdk.proxy2.$Proxy27.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
W20260925 20:00:21.927259 4183 catalog_manager.cc:2209] Already present: failed to create HMS catalog entry for table [id=054453288b4a4834b4bbffcd65e8c2d2]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_daaabef5febc40faa5610a076e6b8b5c already exists)
I20260925 20:00:21.930861 4183 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_cf7560049cf84f22b27dd94a6b4dbcef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:21.937294 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f702fbc80001461298cd1f81e578a05f" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:21.937814 4526 raft_consensus.cc:2468] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:21.938318 4525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d96619f6f7e04b789ba8d09f334ff105" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:21.938588 4525 raft_consensus.cc:2468] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:21.938691 4666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d96619f6f7e04b789ba8d09f334ff105" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:21.939127 4741 leader_election.cc:304] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:21.939204 4666 raft_consensus.cc:2468] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:21.939570 4931 raft_consensus.cc:2804] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:21.939685 4931 raft_consensus.cc:493] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:21.939778 4931 raft_consensus.cc:3060] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:21.940727 4667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f702fbc80001461298cd1f81e578a05f" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:21.941020 4667 raft_consensus.cc:2468] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:21.941625 4739 leader_election.cc:304] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:21.943007 4931 raft_consensus.cc:515] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.943950 4931 leader_election.cc:290] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Requested vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:21.944171 4931 raft_consensus.cc:2804] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:21.944270 4931 raft_consensus.cc:493] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:21.944379 4931 raft_consensus.cc:3060] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:21.947745 4931 raft_consensus.cc:515] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.949106 4525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f702fbc80001461298cd1f81e578a05f" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:21.949349 4525 raft_consensus.cc:3060] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:21.950337 4667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f702fbc80001461298cd1f81e578a05f" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:21.950567 4667 raft_consensus.cc:3060] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:21.952687 4525 raft_consensus.cc:2468] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:21.953177 4741 leader_election.cc:304] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:21.953486 4970 raft_consensus.cc:2804] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:21.953868 4667 raft_consensus.cc:2468] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:21.954222 4525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d96619f6f7e04b789ba8d09f334ff105" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:21.954500 4525 raft_consensus.cc:3060] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:21.957309 4667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d96619f6f7e04b789ba8d09f334ff105" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:21.957616 4667 raft_consensus.cc:3060] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:21.958348 4525 raft_consensus.cc:2468] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:21.958909 4741 leader_election.cc:304] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:21.960811 4667 raft_consensus.cc:2468] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:21.961525 4931 leader_election.cc:290] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Requested vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:21.967048 4931 raft_consensus.cc:2804] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:21.967317 4931 raft_consensus.cc:697] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Becoming Leader. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Running, Role: LEADER
I20260925 20:00:21.968012 4931 consensus_queue.cc:237] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:21.971184 4970 raft_consensus.cc:697] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Becoming Leader. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Running, Role: LEADER
I20260925 20:00:21.971623 4970 consensus_queue.cc:237] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.011883 4180 catalog_manager.cc:2213] created HMS catalog entry for table [id=189081df4c034817b16de973c311b330]
I20260925 20:00:22.036270 4646 tablet_service.cc:1511] Processing CreateTablet for tablet 0cd27940acbb43ef9d75591871c00481 (DEFAULT_TABLE table=default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.036981 4646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cd27940acbb43ef9d75591871c00481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.037072 4970 raft_consensus.cc:493] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:22.037245 4970 raft_consensus.cc:515] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.038023 4970 leader_election.cc:290] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:22.038929 4667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "983d375d3fda4e63ab93b6cde278e187" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:22.039165 4667 raft_consensus.cc:2468] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:22.039664 4739 leader_election.cc:304] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:22.039963 4970 raft_consensus.cc:2804] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:22.040056 4970 raft_consensus.cc:493] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:22.040153 4970 raft_consensus.cc:3060] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.040905 4785 tablet_service.cc:1511] Processing CreateTablet for tablet 0cd27940acbb43ef9d75591871c00481 (DEFAULT_TABLE table=default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.041380 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cd27940acbb43ef9d75591871c00481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.041394 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 0cd27940acbb43ef9d75591871c00481 (DEFAULT_TABLE table=default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.041814 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cd27940acbb43ef9d75591871c00481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.043233 4970 raft_consensus.cc:515] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.044520 4525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "983d375d3fda4e63ab93b6cde278e187" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:22.044654 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "983d375d3fda4e63ab93b6cde278e187" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:22.044773 4525 raft_consensus.cc:2468] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:22.045929 4667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "983d375d3fda4e63ab93b6cde278e187" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:22.046150 4667 raft_consensus.cc:3060] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.049597 4667 raft_consensus.cc:2468] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:22.050275 4739 leader_election.cc:304] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482
I20260925 20:00:22.050789 4931 raft_consensus.cc:2804] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:22.050957 4932 tablet_bootstrap.cc:492] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:22.050979 4931 raft_consensus.cc:697] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Becoming Leader. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Running, Role: LEADER
I20260925 20:00:22.051257 4931 consensus_queue.cc:237] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.052449 4929 tablet_bootstrap.cc:492] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:22.053093 4932 tablet_bootstrap.cc:654] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.053504 4180 catalog_manager.cc:5838] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e reported cstate change: term changed from 0 to 1, leader changed from <none> to dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195). New cstate: current_term: 1 leader_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: HEALTHY } } }
I20260925 20:00:22.054291 4929 tablet_bootstrap.cc:654] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.054688 4180 catalog_manager.cc:5838] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e reported cstate change: term changed from 0 to 1, leader changed from <none> to dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195). New cstate: current_term: 1 leader_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: HEALTHY } } }
I20260925 20:00:22.054916 4970 leader_election.cc:290] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Requested vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:22.055853 4933 tablet_bootstrap.cc:492] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:22.058275 4933 tablet_bootstrap.cc:654] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.066167 4932 tablet_bootstrap.cc:492] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:22.067288 4932 ts_tablet_manager.cc:1403] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.003s
I20260925 20:00:22.068182 4932 raft_consensus.cc:359] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:22.068588 4932 raft_consensus.cc:385] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:22.068717 4932 raft_consensus.cc:740] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:22.069090 4932 consensus_queue.cc:260] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:22.070832 4932 ts_tablet_manager.cc:1434] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
W20260925 20:00:22.079905 4175 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111) [suppressed 12 similar messages]
I20260925 20:00:22.089582 4933 tablet_bootstrap.cc:492] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:22.090018 4933 ts_tablet_manager.cc:1403] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.006s
I20260925 20:00:22.091329 4933 raft_consensus.cc:359] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:22.091735 4933 raft_consensus.cc:385] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:22.091869 4933 raft_consensus.cc:740] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:22.092332 4933 consensus_queue.cc:260] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:22.097501 4933 ts_tablet_manager.cc:1434] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260925 20:00:22.102147 4944 raft_consensus.cc:493] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:22.102342 4944 raft_consensus.cc:515] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.103430 4182 catalog_manager.cc:2213] created HMS catalog entry for table [id=2411a96fbe974a87a2ef047732f0b4bf]
I20260925 20:00:22.104614 4944 leader_election.cc:290] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:22.114969 4182 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48108 (request call id 38) took 1083 ms. Trace:
I20260925 20:00:22.115162 4182 rpcz_store.cc:276] 0925 20:00:21.031852 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:21.032902 (+ 1050us) service_pool.cc:224] Handling call
0925 20:00:21.190471 (+157569us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 20:00:21.191039 (+ 568us) catalog_manager.cc:2186] Created new table and tablet info
0925 20:00:22.103365 (+912326us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 20:00:22.114675 (+ 11310us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 20:00:22.114792 (+ 117us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 20:00:22.114917 (+ 125us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 20:00:22.104260 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 20:00:22.104435 (+ 175us) write_op.cc:432] Acquiring schema lock in shared mode
0925 20:00:22.104441 (+ 6us) write_op.cc:435] Acquired schema lock
0925 20:00:22.104444 (+ 3us) tablet.cc:662] Decoding operations
0925 20:00:22.104538 (+ 94us) write_op.cc:620] Acquiring the partition lock for write op
0925 20:00:22.104553 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0925 20:00:22.104558 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0925 20:00:22.104642 (+ 84us) tablet.cc:701] Row locks acquired
0925 20:00:22.104644 (+ 2us) write_op.cc:260] PREPARE: finished
0925 20:00:22.104685 (+ 41us) write_op.cc:270] Start()
0925 20:00:22.104719 (+ 34us) write_op.cc:276] Timestamp: P: 1790366422104655 usec, L: 0
0925 20:00:22.104723 (+ 4us) op_driver.cc:348] REPLICATION: starting
0925 20:00:22.104911 (+ 188us) log.cc:844] Serialized 432 byte log entry
0925 20:00:22.113750 (+ 8839us) op_driver.cc:464] REPLICATION: finished
0925 20:00:22.113934 (+ 184us) write_op.cc:301] APPLY: starting
0925 20:00:22.113957 (+ 23us) tablet.cc:1394] starting BulkCheckPresence
0925 20:00:22.114014 (+ 57us) tablet.cc:1397] finished BulkCheckPresence
0925 20:00:22.114045 (+ 31us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 20:00:22.114174 (+ 129us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 20:00:22.114215 (+ 41us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 20:00:22.114225 (+ 10us) write_op.cc:312] APPLY: finished
0925 20:00:22.114327 (+ 102us) log.cc:844] Serialized 28 byte log entry
0925 20:00:22.114427 (+ 100us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:22.114452 (+ 25us) write_op.cc:454] Released schema lock
0925 20:00:22.114460 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0925 20:00:22.114545 (+ 85us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:22.114549 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":820589,"Hive Metastore.run_cpu_time_us":348,"Hive Metastore.run_wall_time_us":91449,"mutex_wait_us":9307,"spinlock_wait_cycles":1788288,"child_traces":[["op",{"apply.queue_time_us":115,"apply.run_cpu_time_us":603,"apply.run_wall_time_us":603,"num_ops":2,"prepare.queue_time_us":101,"prepare.run_cpu_time_us":765,"prepare.run_wall_time_us":766,"raft.queue_time_us":432,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":271,"replication_time_us":8986}]]}
I20260925 20:00:22.138927 4646 tablet_service.cc:1511] Processing CreateTablet for tablet 635a8a28ca8444dd8162e29a405810d6 (DEFAULT_TABLE table=default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.139436 4646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 635a8a28ca8444dd8162e29a405810d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.139318 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 635a8a28ca8444dd8162e29a405810d6 (DEFAULT_TABLE table=default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.139928 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 635a8a28ca8444dd8162e29a405810d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.151592 4785 tablet_service.cc:1511] Processing CreateTablet for tablet 635a8a28ca8444dd8162e29a405810d6 (DEFAULT_TABLE table=default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.152364 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 635a8a28ca8444dd8162e29a405810d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.156666 4182 catalog_manager.cc:5838] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e reported cstate change: term changed from 0 to 1, leader changed from <none> to dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195). New cstate: current_term: 1 leader_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: HEALTHY } } }
I20260925 20:00:22.159078 4181 catalog_manager.cc:2213] created HMS catalog entry for table [id=a8f568d7faab469ba7ea8a2a38bfd638]
I20260925 20:00:22.169302 4932 tablet_bootstrap.cc:492] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:22.170130 4181 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48108 (request call id 39) took 1138 ms. Trace:
I20260925 20:00:22.170315 4181 rpcz_store.cc:276] 0925 20:00:21.031962 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:21.032921 (+ 959us) service_pool.cc:224] Handling call
0925 20:00:21.191721 (+158800us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 20:00:21.192314 (+ 593us) catalog_manager.cc:2186] Created new table and tablet info
0925 20:00:22.159013 (+966699us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 20:00:22.169781 (+ 10768us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 20:00:22.169932 (+ 151us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 20:00:22.170074 (+ 142us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 20:00:22.160639 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 20:00:22.160759 (+ 120us) write_op.cc:432] Acquiring schema lock in shared mode
0925 20:00:22.160765 (+ 6us) write_op.cc:435] Acquired schema lock
0925 20:00:22.160768 (+ 3us) tablet.cc:662] Decoding operations
0925 20:00:22.160854 (+ 86us) write_op.cc:620] Acquiring the partition lock for write op
0925 20:00:22.160866 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0925 20:00:22.160870 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0925 20:00:22.160960 (+ 90us) tablet.cc:701] Row locks acquired
0925 20:00:22.160962 (+ 2us) write_op.cc:260] PREPARE: finished
0925 20:00:22.160998 (+ 36us) write_op.cc:270] Start()
0925 20:00:22.161030 (+ 32us) write_op.cc:276] Timestamp: P: 1790366422160965 usec, L: 0
0925 20:00:22.161033 (+ 3us) op_driver.cc:348] REPLICATION: starting
0925 20:00:22.161205 (+ 172us) log.cc:844] Serialized 432 byte log entry
0925 20:00:22.168752 (+ 7547us) op_driver.cc:464] REPLICATION: finished
0925 20:00:22.168946 (+ 194us) write_op.cc:301] APPLY: starting
0925 20:00:22.168967 (+ 21us) tablet.cc:1394] starting BulkCheckPresence
0925 20:00:22.169038 (+ 71us) tablet.cc:1397] finished BulkCheckPresence
0925 20:00:22.169089 (+ 51us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 20:00:22.169287 (+ 198us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 20:00:22.169333 (+ 46us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 20:00:22.169345 (+ 12us) write_op.cc:312] APPLY: finished
0925 20:00:22.169443 (+ 98us) log.cc:844] Serialized 28 byte log entry
0925 20:00:22.169518 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:22.169547 (+ 29us) write_op.cc:454] Released schema lock
0925 20:00:22.169556 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0925 20:00:22.169649 (+ 93us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:22.169655 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":910855,"Hive Metastore.run_cpu_time_us":281,"Hive Metastore.run_wall_time_us":55402,"mutex_wait_us":8091,"spinlock_wait_cycles":1023872,"child_traces":[["op",{"apply.queue_time_us":132,"apply.run_cpu_time_us":690,"apply.run_wall_time_us":689,"num_ops":2,"prepare.queue_time_us":910,"prepare.run_cpu_time_us":666,"prepare.run_wall_time_us":667,"raft.queue_time_us":60,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":7676}]]}
I20260925 20:00:22.171540 4932 tablet_bootstrap.cc:654] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.179116 4181 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_533f83fef17a455883e73ee1c68b252a" } new_table_name: "default.Table_703616ef44ba43f9afb9c4df73eb372c" modify_external_catalogs: true
I20260925 20:00:22.181344 4933 tablet_bootstrap.cc:492] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:22.183946 4933 tablet_bootstrap.cc:654] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.188175 4180 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_c78f9cc1d7e74b51b29858c7d3248fb8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:22.189810 4932 tablet_bootstrap.cc:492] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:22.189999 4932 ts_tablet_manager.cc:1403] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.006s
I20260925 20:00:22.190805 4932 raft_consensus.cc:359] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:22.191020 4932 raft_consensus.cc:385] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:22.191111 4932 raft_consensus.cc:740] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:22.191412 4932 consensus_queue.cc:260] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:22.193432 4932 ts_tablet_manager.cc:1434] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260925 20:00:22.197096 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee843d55757347ef9942dffd917c0ff8" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:22.197345 4526 raft_consensus.cc:2468] T ee843d55757347ef9942dffd917c0ff8 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 0.
I20260925 20:00:22.197924 4933 tablet_bootstrap.cc:492] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:22.198097 4933 ts_tablet_manager.cc:1403] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.004s
I20260925 20:00:22.199014 4933 raft_consensus.cc:359] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:22.199213 4933 raft_consensus.cc:385] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:22.199296 4933 raft_consensus.cc:740] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:22.199518 4933 consensus_queue.cc:260] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:22.199702 4944 raft_consensus.cc:493] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:22.199908 4944 raft_consensus.cc:515] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:22.200757 4944 leader_election.cc:290] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395), 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:22.208827 4646 tablet_service.cc:1511] Processing CreateTablet for tablet 2ddb9fbd0aeb4c4a985508ac67c93227 (DEFAULT_TABLE table=default.table_7722a249d10f4616ba5b5bb2fc149c18 [id=a8f568d7faab469ba7ea8a2a38bfd638]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.209511 4646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ddb9fbd0aeb4c4a985508ac67c93227. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.213804 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 2ddb9fbd0aeb4c4a985508ac67c93227 (DEFAULT_TABLE table=default.table_7722a249d10f4616ba5b5bb2fc149c18 [id=a8f568d7faab469ba7ea8a2a38bfd638]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.214300 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ddb9fbd0aeb4c4a985508ac67c93227. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.220098 4932 tablet_bootstrap.cc:492] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:22.221979 4932 tablet_bootstrap.cc:654] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.223619 4933 ts_tablet_manager.cc:1434] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.025s user 0.000s sys 0.002s
I20260925 20:00:22.224831 4784 tablet_service.cc:1511] Processing CreateTablet for tablet 2ddb9fbd0aeb4c4a985508ac67c93227 (DEFAULT_TABLE table=default.table_7722a249d10f4616ba5b5bb2fc149c18 [id=a8f568d7faab469ba7ea8a2a38bfd638]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.225440 4784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ddb9fbd0aeb4c4a985508ac67c93227. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.228117 4932 tablet_bootstrap.cc:492] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:22.228770 4932 ts_tablet_manager.cc:1403] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260925 20:00:22.229588 4932 raft_consensus.cc:359] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.229799 4932 raft_consensus.cc:385] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:22.229887 4932 raft_consensus.cc:740] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:22.231128 4931 raft_consensus.cc:493] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:22.231303 4931 raft_consensus.cc:515] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.232019 4931 leader_election.cc:290] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:22.230144 4932 consensus_queue.cc:260] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.232769 4667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee843d55757347ef9942dffd917c0ff8" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:22.232894 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee843d55757347ef9942dffd917c0ff8" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:22.233057 4932 ts_tablet_manager.cc:1434] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260925 20:00:22.233040 4667 raft_consensus.cc:2468] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:22.233114 4526 raft_consensus.cc:2468] T ee843d55757347ef9942dffd917c0ff8 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:22.233724 4741 leader_election.cc:304] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:22.234059 4931 raft_consensus.cc:2804] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:22.234171 4931 raft_consensus.cc:493] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:22.234261 4931 raft_consensus.cc:3060] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.237758 4931 raft_consensus.cc:515] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.238538 4931 leader_election.cc:290] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Requested vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:22.239323 4667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee843d55757347ef9942dffd917c0ff8" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:22.239424 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee843d55757347ef9942dffd917c0ff8" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:22.239564 4667 raft_consensus.cc:3060] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.239626 4526 raft_consensus.cc:3060] T ee843d55757347ef9942dffd917c0ff8 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.242444 4667 raft_consensus.cc:2468] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:22.242887 4526 raft_consensus.cc:2468] T ee843d55757347ef9942dffd917c0ff8 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:22.243364 4739 leader_election.cc:304] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:22.244582 4931 raft_consensus.cc:2804] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:22.245764 4182 master_service.cc:960] ReplaceTablet: received request to replace tablet f702fbc80001461298cd1f81e578a05f from {username='slave'} at 127.0.0.1:48108
I20260925 20:00:22.251976 4933 tablet_bootstrap.cc:492] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:22.252665 4929 tablet_bootstrap.cc:492] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:22.252880 4929 ts_tablet_manager.cc:1403] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.200s user 0.041s sys 0.052s
I20260925 20:00:22.253762 4929 raft_consensus.cc:359] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:22.253940 4933 tablet_bootstrap.cc:654] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.253829 4931 raft_consensus.cc:697] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Becoming Leader. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Running, Role: LEADER
I20260925 20:00:22.254014 4929 raft_consensus.cc:385] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:22.254099 4929 raft_consensus.cc:740] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:22.254310 4931 consensus_queue.cc:237] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.254380 4929 consensus_queue.cc:260] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:22.254937 4929 ts_tablet_manager.cc:1434] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260925 20:00:22.255868 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd27940acbb43ef9d75591871c00481" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:22.256108 4526 raft_consensus.cc:2468] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 0.
I20260925 20:00:22.260118 4603 leader_election.cc:304] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:22.262377 4929 tablet_bootstrap.cc:492] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:22.265654 4182 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Sending DeleteTablet for 3 replicas of tablet f702fbc80001461298cd1f81e578a05f
I20260925 20:00:22.269443 4929 tablet_bootstrap.cc:654] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.270744 4983 raft_consensus.cc:2764] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260925 20:00:22.270768 4603 leader_election.cc:304] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:22.270903 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet f702fbc80001461298cd1f81e578a05f with delete_type TABLET_DATA_DELETED (replaced by tablet f4b5baba575d4538b807a69eff0d90dd) from {username='slave'} at 127.0.0.1:54948
I20260925 20:00:22.271075 4983 raft_consensus.cc:2804] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:22.271193 4983 raft_consensus.cc:493] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:22.271319 4983 raft_consensus.cc:3060] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.273882 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee843d55757347ef9942dffd917c0ff8" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" is_pre_election: true
I20260925 20:00:22.274726 4804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd27940acbb43ef9d75591871c00481" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" is_pre_election: true
I20260925 20:00:22.275067 4804 raft_consensus.cc:2468] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 0.
I20260925 20:00:22.275285 4983 raft_consensus.cc:515] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:22.276050 4182 catalog_manager.cc:6934] ReplaceTablet: tablet f702fbc80001461298cd1f81e578a05f deleted and replaced by tablet f4b5baba575d4538b807a69eff0d90dd
I20260925 20:00:22.276264 4983 leader_election.cc:290] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 election: Requested vote from peers dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395), 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:22.267300 4646 tablet_service.cc:1558] Processing DeleteTablet for tablet f702fbc80001461298cd1f81e578a05f with delete_type TABLET_DATA_DELETED (replaced by tablet f4b5baba575d4538b807a69eff0d90dd) from {username='slave'} at 127.0.0.1:57378
I20260925 20:00:22.269738 4933 tablet_bootstrap.cc:492] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:22.276640 4933 ts_tablet_manager.cc:1403] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.007s
I20260925 20:00:22.277113 4784 tablet_service.cc:1558] Processing DeleteTablet for tablet f702fbc80001461298cd1f81e578a05f with delete_type TABLET_DATA_DELETED (replaced by tablet f4b5baba575d4538b807a69eff0d90dd) from {username='slave'} at 127.0.0.1:54514
I20260925 20:00:22.277462 4933 raft_consensus.cc:359] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.277653 4933 raft_consensus.cc:385] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:22.277729 4933 raft_consensus.cc:740] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:22.277971 4933 consensus_queue.cc:260] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.278501 4933 ts_tablet_manager.cc:1434] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260925 20:00:22.300730 4940 raft_consensus.cc:493] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:22.300994 4940 raft_consensus.cc:515] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.303462 4940 leader_election.cc:290] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:22.310266 4647 tablet_service.cc:1511] Processing CreateTablet for tablet f4b5baba575d4538b807a69eff0d90dd (DEFAULT_TABLE table=default.table_1c5a83ea353b453da40b24de92d7e850 [id=90221d8ab3b94046bbdff933844c7f22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.310770 4647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4b5baba575d4538b807a69eff0d90dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.313328 4989 tablet_replica.cc:333] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:22.313606 4989 raft_consensus.cc:2243] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:22.313889 4989 raft_consensus.cc:2272] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:22.314540 4989 ts_tablet_manager.cc:1916] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:22.316216 4667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "460399668df444c5af242442af590e1a" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:22.316534 4667 raft_consensus.cc:2468] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:22.317531 4455 leader_election.cc:304] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:22.319556 4940 raft_consensus.cc:2804] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:22.319728 4940 raft_consensus.cc:493] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:22.319846 4940 raft_consensus.cc:3060] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.325139 4506 tablet_service.cc:1511] Processing CreateTablet for tablet f4b5baba575d4538b807a69eff0d90dd (DEFAULT_TABLE table=default.table_1c5a83ea353b453da40b24de92d7e850 [id=90221d8ab3b94046bbdff933844c7f22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.325821 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4b5baba575d4538b807a69eff0d90dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.329465 4804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "460399668df444c5af242442af590e1a" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" is_pre_election: true
I20260925 20:00:22.329741 4804 raft_consensus.cc:2468] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:22.336578 4932 tablet_bootstrap.cc:492] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:22.338840 4932 tablet_bootstrap.cc:654] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.340504 4983 raft_consensus.cc:493] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:22.340727 4983 raft_consensus.cc:515] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.341569 4983 leader_election.cc:290] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:22.339555 4940 raft_consensus.cc:515] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.342192 4785 tablet_service.cc:1511] Processing CreateTablet for tablet f4b5baba575d4538b807a69eff0d90dd (DEFAULT_TABLE table=default.table_1c5a83ea353b453da40b24de92d7e850 [id=90221d8ab3b94046bbdff933844c7f22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.342693 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4b5baba575d4538b807a69eff0d90dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.343066 4667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "460399668df444c5af242442af590e1a" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:22.343286 4667 raft_consensus.cc:3060] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.344554 4933 tablet_bootstrap.cc:492] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:22.346880 4933 tablet_bootstrap.cc:654] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.346959 4667 raft_consensus.cc:2468] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
I20260925 20:00:22.347591 4455 leader_election.cc:304] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:22.356003 4182 catalog_manager.cc:5838] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e reported cstate change: term changed from 0 to 1, leader changed from <none> to dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195). New cstate: current_term: 1 leader_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: HEALTHY } } }
I20260925 20:00:22.362494 4804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd27940acbb43ef9d75591871c00481" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
I20260925 20:00:22.362780 4804 raft_consensus.cc:3060] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.364724 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "460399668df444c5af242442af590e1a" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
I20260925 20:00:22.364996 4805 raft_consensus.cc:3060] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.366559 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd27940acbb43ef9d75591871c00481" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:22.366911 4526 raft_consensus.cc:3060] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.368427 4525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff315b3b8e64ed0838b6241eeb442d3" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:22.368666 4525 raft_consensus.cc:2468] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 0.
I20260925 20:00:22.369693 4940 leader_election.cc:290] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Requested vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:22.369772 4803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff315b3b8e64ed0838b6241eeb442d3" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" is_pre_election: true
I20260925 20:00:22.370055 4803 raft_consensus.cc:2468] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 0.
I20260925 20:00:22.370128 4940 raft_consensus.cc:2804] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:22.370258 4940 raft_consensus.cc:697] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 LEADER]: Becoming Leader. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Running, Role: LEADER
I20260925 20:00:22.370793 4940 consensus_queue.cc:237] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.373674 4603 leader_election.cc:304] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:22.378361 4983 raft_consensus.cc:2804] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:22.378544 4983 raft_consensus.cc:493] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:22.378669 4983 raft_consensus.cc:3060] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.388015 4983 raft_consensus.cc:515] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.388593 4989 ts_tablet_manager.cc:1929] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 20:00:22.388736 4989 log.cc:1199] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/f702fbc80001461298cd1f81e578a05f
I20260925 20:00:22.389133 4989 ts_tablet_manager.cc:1950] T f702fbc80001461298cd1f81e578a05f P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:22.389724 4804 raft_consensus.cc:2468] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 1.
I20260925 20:00:22.390529 4526 raft_consensus.cc:2468] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 1.
I20260925 20:00:22.391211 4805 raft_consensus.cc:2468] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
I20260925 20:00:22.392668 4174 catalog_manager.cc:5063] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): tablet f702fbc80001461298cd1f81e578a05f (table default.table_1c5a83ea353b453da40b24de92d7e850 [id=90221d8ab3b94046bbdff933844c7f22]) successfully deleted
I20260925 20:00:22.393905 4983 leader_election.cc:290] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 election: Requested vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
W20260925 20:00:22.403555 4455 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.199.194:36647, user_credentials={real_user=slave}} blocked reactor thread for 55948.8us
I20260925 20:00:22.405414 4933 tablet_bootstrap.cc:492] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:22.405597 4933 ts_tablet_manager.cc:1403] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.061s user 0.003s sys 0.011s
I20260925 20:00:22.406473 4933 raft_consensus.cc:359] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.406706 4933 raft_consensus.cc:385] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:22.407290 4933 raft_consensus.cc:740] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:22.407625 4933 consensus_queue.cc:260] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.410984 4182 master_service.cc:960] ReplaceTablet: received request to replace tablet d96619f6f7e04b789ba8d09f334ff105 from {username='slave'} at 127.0.0.1:48108
I20260925 20:00:22.412868 4933 ts_tablet_manager.cc:1434] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.007s user 0.002s sys 0.002s
I20260925 20:00:22.417160 4988 tablet_replica.cc:333] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:22.417586 4988 raft_consensus.cc:2243] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:22.417953 4988 raft_consensus.cc:2272] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:22.418792 4988 ts_tablet_manager.cc:1916] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:22.422653 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff315b3b8e64ed0838b6241eeb442d3" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:22.422991 4526 raft_consensus.cc:3060] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.427647 4526 raft_consensus.cc:2468] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 1.
I20260925 20:00:22.431193 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff315b3b8e64ed0838b6241eeb442d3" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
I20260925 20:00:22.431491 4805 raft_consensus.cc:3060] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.435871 4805 raft_consensus.cc:2468] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 1.
I20260925 20:00:22.436838 4603 leader_election.cc:304] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:22.437508 4983 raft_consensus.cc:2804] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:22.437686 4983 raft_consensus.cc:697] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 LEADER]: Becoming Leader. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Running, Role: LEADER
I20260925 20:00:22.438241 4983 consensus_queue.cc:237] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:22.443626 4990 tablet_replica.cc:333] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:22.443979 4990 raft_consensus.cc:2243] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:22.444836 4990 pending_rounds.cc:70] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e: Trying to abort 1 pending ops.
I20260925 20:00:22.445290 4990 pending_rounds.cc:77] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333340864399159296 op_type: NO_OP noop_request { }
I20260925 20:00:22.445624 4990 raft_consensus.cc:2889] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 20:00:22.445850 4990 raft_consensus.cc:2272] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:22.446403 4603 leader_election.cc:304] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:22.446842 4983 raft_consensus.cc:2804] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:22.447933 4983 raft_consensus.cc:697] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 LEADER]: Becoming Leader. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Running, Role: LEADER
I20260925 20:00:22.448397 4983 consensus_queue.cc:237] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.452093 4990 ts_tablet_manager.cc:1916] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:22.453934 4929 tablet_bootstrap.cc:492] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:22.454111 4929 ts_tablet_manager.cc:1403] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.196s user 0.014s sys 0.044s
I20260925 20:00:22.455453 4929 raft_consensus.cc:359] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:22.455662 4929 raft_consensus.cc:385] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:22.455750 4929 raft_consensus.cc:740] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:22.456019 4929 consensus_queue.cc:260] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:22.456681 4929 ts_tablet_manager.cc:1434] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 20:00:22.457078 4929 tablet_bootstrap.cc:492] T 2ddb9fbd0aeb4c4a985508ac67c93227 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:22.459182 4929 tablet_bootstrap.cc:654] T 2ddb9fbd0aeb4c4a985508ac67c93227 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.468138 4932 tablet_bootstrap.cc:492] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:22.468400 4932 ts_tablet_manager.cc:1403] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.132s user 0.015s sys 0.028s
I20260925 20:00:22.469506 4932 raft_consensus.cc:359] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.469784 4932 raft_consensus.cc:385] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:22.469887 4932 raft_consensus.cc:740] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:22.470155 4932 consensus_queue.cc:260] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.470758 4932 ts_tablet_manager.cc:1434] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 20:00:22.476629 4990 ts_tablet_manager.cc:1929] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:22.476904 4990 log.cc:1199] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/f702fbc80001461298cd1f81e578a05f
I20260925 20:00:22.477463 4990 ts_tablet_manager.cc:1950] T f702fbc80001461298cd1f81e578a05f P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:22.443193 4187 catalog_manager.cc:3541] renamed table default.table_da698ad34fc34a51ae83565db0a9ff48 [id=c3a1c6d72e6b46c4b253c1c356b17bd7] in HMS: new name default.table_c860d14624744708bf4ff50a84517b59
I20260925 20:00:22.480906 4176 catalog_manager.cc:5063] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395): tablet f702fbc80001461298cd1f81e578a05f (table default.table_1c5a83ea353b453da40b24de92d7e850 [id=90221d8ab3b94046bbdff933844c7f22]) successfully deleted
I20260925 20:00:22.507165 4182 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Sending DeleteTablet for 3 replicas of tablet d96619f6f7e04b789ba8d09f334ff105
I20260925 20:00:22.508522 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet d96619f6f7e04b789ba8d09f334ff105 with delete_type TABLET_DATA_DELETED (replaced by tablet 88f6f526af054da1973d202e14ce22ae) from {username='slave'} at 127.0.0.1:54948
I20260925 20:00:22.510087 4647 tablet_service.cc:1558] Processing DeleteTablet for tablet d96619f6f7e04b789ba8d09f334ff105 with delete_type TABLET_DATA_DELETED (replaced by tablet 88f6f526af054da1973d202e14ce22ae) from {username='slave'} at 127.0.0.1:57378
I20260925 20:00:22.510092 4784 tablet_service.cc:1558] Processing DeleteTablet for tablet d96619f6f7e04b789ba8d09f334ff105 with delete_type TABLET_DATA_DELETED (replaced by tablet 88f6f526af054da1973d202e14ce22ae) from {username='slave'} at 127.0.0.1:54514
I20260925 20:00:22.510397 4989 tablet_replica.cc:333] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:22.510617 4989 raft_consensus.cc:2243] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:22.510812 4989 raft_consensus.cc:2272] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:22.511825 4989 ts_tablet_manager.cc:1916] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:22.516498 4990 tablet_replica.cc:333] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:22.516757 4990 raft_consensus.cc:2243] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:22.517045 4990 pending_rounds.cc:70] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e: Trying to abort 1 pending ops.
I20260925 20:00:22.517112 4990 pending_rounds.cc:77] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333340864387600384 op_type: NO_OP noop_request { }
I20260925 20:00:22.517257 4990 raft_consensus.cc:2889] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 20:00:22.517370 4990 raft_consensus.cc:2272] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:22.518311 4990 ts_tablet_manager.cc:1916] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:22.522996 4647 tablet_service.cc:1511] Processing CreateTablet for tablet 88f6f526af054da1973d202e14ce22ae (DEFAULT_TABLE table=default.table_44dac2dae5044f0d8bc0da4a5aafa446 [id=78f973c6a62948e18d7f86d475b64812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.523447 4647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88f6f526af054da1973d202e14ce22ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.523734 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 88f6f526af054da1973d202e14ce22ae (DEFAULT_TABLE table=default.table_44dac2dae5044f0d8bc0da4a5aafa446 [id=78f973c6a62948e18d7f86d475b64812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.524420 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88f6f526af054da1973d202e14ce22ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.528798 4784 tablet_service.cc:1511] Processing CreateTablet for tablet 88f6f526af054da1973d202e14ce22ae (DEFAULT_TABLE table=default.table_44dac2dae5044f0d8bc0da4a5aafa446 [id=78f973c6a62948e18d7f86d475b64812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.529307 4784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88f6f526af054da1973d202e14ce22ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.532730 4933 tablet_bootstrap.cc:492] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:22.534698 4933 tablet_bootstrap.cc:654] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.536041 4932 tablet_bootstrap.cc:492] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:22.536010 4989 ts_tablet_manager.cc:1929] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 20:00:22.536192 4989 log.cc:1199] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/d96619f6f7e04b789ba8d09f334ff105
I20260925 20:00:22.536530 4182 catalog_manager.cc:6934] ReplaceTablet: tablet d96619f6f7e04b789ba8d09f334ff105 deleted and replaced by tablet 88f6f526af054da1973d202e14ce22ae
I20260925 20:00:22.537995 4932 tablet_bootstrap.cc:654] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.539217 4990 ts_tablet_manager.cc:1929] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:22.550716 4988 ts_tablet_manager.cc:1929] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 20:00:22.550899 4988 log.cc:1199] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/f702fbc80001461298cd1f81e578a05f
I20260925 20:00:22.551362 4988 ts_tablet_manager.cc:1950] T f702fbc80001461298cd1f81e578a05f P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:22.551548 4932 tablet_bootstrap.cc:492] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:22.551695 4989 ts_tablet_manager.cc:1950] T d96619f6f7e04b789ba8d09f334ff105 P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:22.551733 4932 ts_tablet_manager.cc:1403] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.003s
I20260925 20:00:22.551890 4990 log.cc:1199] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/d96619f6f7e04b789ba8d09f334ff105
I20260925 20:00:22.552870 4176 catalog_manager.cc:5063] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): tablet f702fbc80001461298cd1f81e578a05f (table default.table_1c5a83ea353b453da40b24de92d7e850 [id=90221d8ab3b94046bbdff933844c7f22]) successfully deleted
I20260925 20:00:22.552798 4932 raft_consensus.cc:359] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.553041 4932 raft_consensus.cc:385] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:22.553126 4932 raft_consensus.cc:740] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:22.553392 4932 consensus_queue.cc:260] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.553498 4933 tablet_bootstrap.cc:492] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:22.553665 4933 ts_tablet_manager.cc:1403] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.005s
I20260925 20:00:22.553922 4932 ts_tablet_manager.cc:1434] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260925 20:00:22.550021 4929 tablet_bootstrap.cc:492] T 2ddb9fbd0aeb4c4a985508ac67c93227 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:22.554255 4929 ts_tablet_manager.cc:1403] T 2ddb9fbd0aeb4c4a985508ac67c93227 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.097s user 0.008s sys 0.061s
I20260925 20:00:22.554466 4933 raft_consensus.cc:359] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.554677 4933 raft_consensus.cc:385] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:22.554751 4933 raft_consensus.cc:740] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:22.555018 4933 consensus_queue.cc:260] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.555068 4929 raft_consensus.cc:359] T 2ddb9fbd0aeb4c4a985508ac67c93227 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.555260 4929 raft_consensus.cc:385] T 2ddb9fbd0aeb4c4a985508ac67c93227 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:22.555338 4929 raft_consensus.cc:740] T 2ddb9fbd0aeb4c4a985508ac67c93227 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:22.555589 4933 ts_tablet_manager.cc:1434] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260925 20:00:22.555660 4929 consensus_queue.cc:260] T 2ddb9fbd0aeb4c4a985508ac67c93227 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.557495 4929 ts_tablet_manager.cc:1434] T 2ddb9fbd0aeb4c4a985508ac67c93227 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260925 20:00:22.557812 4929 tablet_bootstrap.cc:492] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:22.558642 4990 ts_tablet_manager.cc:1950] T d96619f6f7e04b789ba8d09f334ff105 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
W20260925 20:00:22.560191 4936 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
I20260925 20:00:22.560545 4983 raft_consensus.cc:493] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 20:00:22.560667 4936 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260925 20:00:22.560710 4983 raft_consensus.cc:515] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
W20260925 20:00:22.560814 4936 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260925 20:00:22.561414 4983 leader_election.cc:290] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:22.562218 4182 catalog_manager.cc:5838] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 reported cstate change: term changed from 0 to 1, leader changed from <none> to 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193). New cstate: current_term: 1 leader_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: UNKNOWN } } }
I20260925 20:00:22.559965 4929 tablet_bootstrap.cc:654] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.563499 4988 tablet_replica.cc:333] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:22.563807 4988 raft_consensus.cc:2243] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:22.564085 4988 raft_consensus.cc:2272] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:22.564742 4988 ts_tablet_manager.cc:1916] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:22.565748 4174 catalog_manager.cc:5063] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): tablet d96619f6f7e04b789ba8d09f334ff105 (table default.table_44dac2dae5044f0d8bc0da4a5aafa446 [id=78f973c6a62948e18d7f86d475b64812]) successfully deleted
I20260925 20:00:22.566164 4176 catalog_manager.cc:5063] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395): tablet d96619f6f7e04b789ba8d09f334ff105 (table default.table_44dac2dae5044f0d8bc0da4a5aafa446 [id=78f973c6a62948e18d7f86d475b64812]) successfully deleted
I20260925 20:00:22.569545 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddb9fbd0aeb4c4a985508ac67c93227" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:22.569797 4526 raft_consensus.cc:2468] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 0.
I20260925 20:00:22.570399 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddb9fbd0aeb4c4a985508ac67c93227" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" is_pre_election: true
I20260925 20:00:22.570632 4805 raft_consensus.cc:2468] T 2ddb9fbd0aeb4c4a985508ac67c93227 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 0.
I20260925 20:00:22.572022 4988 ts_tablet_manager.cc:1929] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 20:00:22.572235 4988 log.cc:1199] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/d96619f6f7e04b789ba8d09f334ff105
I20260925 20:00:22.573192 4988 ts_tablet_manager.cc:1950] T d96619f6f7e04b789ba8d09f334ff105 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:22.574451 4176 catalog_manager.cc:5063] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): tablet d96619f6f7e04b789ba8d09f334ff105 (table default.table_44dac2dae5044f0d8bc0da4a5aafa446 [id=78f973c6a62948e18d7f86d475b64812]) successfully deleted
I20260925 20:00:22.574611 4603 leader_election.cc:304] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:22.575093 4983 raft_consensus.cc:2804] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:22.575253 4983 raft_consensus.cc:493] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:22.575352 4983 raft_consensus.cc:3060] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.577518 4983 raft_consensus.cc:515] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.578250 4983 leader_election.cc:290] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 election: Requested vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:22.578948 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddb9fbd0aeb4c4a985508ac67c93227" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
I20260925 20:00:22.579170 4805 raft_consensus.cc:3060] T 2ddb9fbd0aeb4c4a985508ac67c93227 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.581256 4182 catalog_manager.cc:5838] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194). New cstate: current_term: 1 leader_uuid: "46a6f571dfa749b7905d8458d4c929b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: HEALTHY } } }
I20260925 20:00:22.581789 4182 catalog_manager.cc:5838] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194). New cstate: current_term: 1 leader_uuid: "46a6f571dfa749b7905d8458d4c929b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: UNKNOWN } } }
I20260925 20:00:22.582062 4805 raft_consensus.cc:2468] T 2ddb9fbd0aeb4c4a985508ac67c93227 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 1.
I20260925 20:00:22.588040 4929 tablet_bootstrap.cc:492] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:22.588217 4929 ts_tablet_manager.cc:1403] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.004s
I20260925 20:00:22.588838 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddb9fbd0aeb4c4a985508ac67c93227" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:22.589051 4526 raft_consensus.cc:3060] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.589088 4929 raft_consensus.cc:359] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.589264 4929 raft_consensus.cc:385] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:22.589365 4929 raft_consensus.cc:740] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:22.589665 4929 consensus_queue.cc:260] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.590293 4929 ts_tablet_manager.cc:1434] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 20:00:22.592396 4929 tablet_bootstrap.cc:492] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:22.592588 4526 raft_consensus.cc:2468] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 1.
I20260925 20:00:22.593628 4603 leader_election.cc:304] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:22.593986 4983 raft_consensus.cc:2804] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:22.594128 4983 raft_consensus.cc:697] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 LEADER]: Becoming Leader. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Running, Role: LEADER
I20260925 20:00:22.594259 4929 tablet_bootstrap.cc:654] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.594432 4983 consensus_queue.cc:237] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.610767 4182 catalog_manager.cc:5838] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194). New cstate: current_term: 1 leader_uuid: "46a6f571dfa749b7905d8458d4c929b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: UNKNOWN } } }
I20260925 20:00:22.612797 4929 tablet_bootstrap.cc:492] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:22.612990 4929 ts_tablet_manager.cc:1403] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.001s
I20260925 20:00:22.613785 4929 raft_consensus.cc:359] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.613991 4929 raft_consensus.cc:385] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:22.614066 4929 raft_consensus.cc:740] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:22.614312 4929 consensus_queue.cc:260] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
W20260925 20:00:22.615226 4936 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 20:00:22.615332 4936 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 20:00:22.615422 4936 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 20:00:22.615483 4936 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 20:00:22.618578 4929 ts_tablet_manager.cc:1434] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260925 20:00:22.624609 4182 master_service.cc:960] ReplaceTablet: received request to replace tablet 2a8e9b9011114ee4b7b364870ef864b4 from {username='slave'} at 127.0.0.1:48108
I20260925 20:00:22.625100 4182 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48108 (request call id 57) took 1050 ms. Trace:
I20260925 20:00:22.625286 4182 rpcz_store.cc:276] 0925 20:00:21.574544 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:22.624543 (+1049999us) service_pool.cc:224] Handling call
0925 20:00:22.625043 (+ 500us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:22.626305 4182 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_4ec42ee954664a658b3aad36206c53aa" } modify_external_catalogs: true
I20260925 20:00:22.626648 4182 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_4ec42ee954664a658b3aad36206c53aa" } modify_external_catalogs: true
I20260925 20:00:22.640013 4936 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.199.212:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.253:41897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.199.253
--webserver_port=41541
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 25 Sep 2026 19:43:21 UTC on bdcb31816ec0
build id 13997
ASAN enabled
I20260925 20:00:22.642335 4936 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 20:00:22.642644 4983 raft_consensus.cc:493] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:22.642849 4983 raft_consensus.cc:515] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:22.643656 4983 leader_election.cc:290] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395), 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:22.644729 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635a8a28ca8444dd8162e29a405810d6" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:22.644994 4936 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260925 20:00:22.644980 4526 raft_consensus.cc:2468] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 0.
I20260925 20:00:22.645524 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635a8a28ca8444dd8162e29a405810d6" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" is_pre_election: true
I20260925 20:00:22.645751 4805 raft_consensus.cc:2468] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 0.
I20260925 20:00:22.646301 4603 leader_election.cc:304] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:22.647087 4983 raft_consensus.cc:2804] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:22.647184 4983 raft_consensus.cc:493] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:22.647285 4983 raft_consensus.cc:3060] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.650236 4983 raft_consensus.cc:515] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:22.651432 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635a8a28ca8444dd8162e29a405810d6" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
I20260925 20:00:22.651643 4805 raft_consensus.cc:3060] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.652583 4185 catalog_manager.cc:3541] renamed table default.table_44dac2dae5044f0d8bc0da4a5aafa446 [id=78f973c6a62948e18d7f86d475b64812] in HMS: new name default.table_e72a162b7a554904a9ca0cceb5e6b273
I20260925 20:00:22.655750 4805 raft_consensus.cc:2468] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 1.
I20260925 20:00:22.656656 4603 leader_election.cc:304] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:22.657049 4944 raft_consensus.cc:2804] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:22.663314 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635a8a28ca8444dd8162e29a405810d6" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:22.663655 4526 raft_consensus.cc:3060] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.677628 4667 raft_consensus.cc:1275] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Refusing update from remote peer dd2623c3b2c3497ca1df4a676fa6807e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 20:00:22.677838 4525 raft_consensus.cc:3060] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.680894 4931 consensus_queue.cc:1048] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:22.682263 4526 raft_consensus.cc:2468] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 1.
I20260925 20:00:22.683291 4983 leader_election.cc:290] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 election: Requested vote from peers dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395), 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:22.691181 4944 raft_consensus.cc:697] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 LEADER]: Becoming Leader. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Running, Role: LEADER
I20260925 20:00:22.691834 4944 consensus_queue.cc:237] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
W20260925 20:00:22.695900 4403 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 88f6f526af054da1973d202e14ce22ae consensus state has no leader
I20260925 20:00:22.682626 4525 raft_consensus.cc:1275] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Refusing update from remote peer dd2623c3b2c3497ca1df4a676fa6807e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 20:00:22.700594 4931 consensus_queue.cc:1048] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 20:00:22.708344 5012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:22.711460 5016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 20:00:22.719159 5009 mvcc.cc:204] Tried to move back new op lower bound from 7333340867278626816 to 7333340864724697088. Current Snapshot: MvccSnapshot[applied={T|T < 7333340867278626816}]
I20260925 20:00:22.719444 5021 tablet.cc:1751] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 20:00:22.722138 5018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 20:00:22.727984 5022 tablet.cc:1751] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 20:00:22.738531 4173 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 88f6f526af054da1973d202e14ce22ae (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 88f6f526af054da1973d202e14ce22ae consensus state has no leader
I20260925 20:00:22.739362 4936 server_base.cc:1061] running on GCE node
I20260925 20:00:22.740760 4936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 20:00:22.742794 5023 tablet.cc:1751] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 20:00:22.745092 4936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 20:00:22.746417 4936 hybrid_clock.cc:648] HybridClock initialized: now 1790366422746336 us; error 69 us; skew 500 ppm
I20260925 20:00:22.746893 4936 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 20:00:22.763489 4936 webserver.cc:533] Webserver started at http://127.3.199.253:41541/ using document root <none> and password file <none>
I20260925 20:00:22.764935 4936 fs_manager.cc:362] Metadata directory not provided
I20260925 20:00:22.765141 4936 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 20:00:22.784631 4936 fs_manager.cc:714] Time spent opening directory manager: real 0.016s user 0.012s sys 0.004s
I20260925 20:00:22.798424 5028 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:22.811024 4936 fs_manager.cc:730] Time spent opening block manager: real 0.024s user 0.004s sys 0.007s
I20260925 20:00:22.811305 4936 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data,/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
uuid: "da29ee550f82456fa7c5ce7ea916909f"
format_stamp: "Formatted at 2026-09-25 20:00:17 on dist-test-slave-9gwk"
I20260925 20:00:22.812443 4936 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 20:00:22.817775 4189 catalog_manager.cc:3541] renamed table default.table_1c5a83ea353b453da40b24de92d7e850 [id=90221d8ab3b94046bbdff933844c7f22] in HMS: new name default.table_b74d63554f344195800ce065c2a3cddf
W20260925 20:00:22.820649 4184 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_08807903f9884f2a9707d4066e7f2f52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 20:00:22.820773 4183 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_cf7560049cf84f22b27dd94a6b4dbcef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 20:00:22.821404 4187 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48108 (request call id 41) took 1720 ms. Trace:
I20260925 20:00:22.821547 4187 rpcz_store.cc:276] 0925 20:00:21.100603 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:21.104268 (+ 3665us) service_pool.cc:224] Handling call
0925 20:00:21.363114 (+258846us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 20:00:22.820561 (+1457447us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 20:00:22.820958 (+ 397us) catalog_manager.cc:3764] Apply alter schema
0925 20:00:22.821348 (+ 390us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":795415,"Hive Metastore.run_cpu_time_us":1177,"Hive Metastore.run_wall_time_us":284124}
W20260925 20:00:22.821826 4180 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_c78f9cc1d7e74b51b29858c7d3248fb8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 20:00:22.822122 4185 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48108 (request call id 42) took 1643 ms. Trace:
I20260925 20:00:22.822242 4185 rpcz_store.cc:276] 0925 20:00:21.178243 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:21.182816 (+ 4573us) service_pool.cc:224] Handling call
0925 20:00:21.363114 (+180298us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 20:00:22.821584 (+1458470us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 20:00:22.821834 (+ 250us) catalog_manager.cc:3764] Apply alter schema
0925 20:00:22.822078 (+ 244us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":1079594,"Hive Metastore.run_cpu_time_us":789,"Hive Metastore.run_wall_time_us":209390,"spinlock_wait_cycles":3328}
W20260925 20:00:22.824568 4188 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_30a6c0d501b04bfc882cec2f43c06f83 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 20:00:22.825944 4187 catalog_manager.cc:5838] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194). New cstate: current_term: 1 leader_uuid: "46a6f571dfa749b7905d8458d4c929b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: HEALTHY } } }
I20260925 20:00:22.827090 4187 catalog_manager.cc:5938] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647) does not have the latest schema for tablet 983d375d3fda4e63ab93b6cde278e187 (table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]). Expected version 1 got 0
W20260925 20:00:22.841691 4403 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f4b5baba575d4538b807a69eff0d90dd consensus state has no leader
I20260925 20:00:22.843044 4189 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48108 (request call id 48) took 1545 ms. Trace:
I20260925 20:00:22.843187 4189 rpcz_store.cc:276] 0925 20:00:21.297229 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:21.433345 (+136116us) service_pool.cc:224] Handling call
0925 20:00:22.697162 (+1263817us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 20:00:22.842318 (+145156us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 20:00:22.842612 (+ 294us) catalog_manager.cc:3764] Apply alter schema
0925 20:00:22.842954 (+ 342us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":2950,"Hive Metastore.run_cpu_time_us":992,"Hive Metastore.run_wall_time_us":117347}
I20260925 20:00:22.847476 4185 catalog_manager.cc:5938] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741) does not have the latest schema for tablet 983d375d3fda4e63ab93b6cde278e187 (table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]). Expected version 1 got 0
I20260925 20:00:22.848798 4185 master_service.cc:960] ReplaceTablet: received request to replace tablet 460399668df444c5af242442af590e1a from {username='slave'} at 127.0.0.1:48108
I20260925 20:00:22.861420 4185 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Sending DeleteTablet for 3 replicas of tablet 460399668df444c5af242442af590e1a
I20260925 20:00:22.864233 4647 tablet_service.cc:1558] Processing DeleteTablet for tablet 460399668df444c5af242442af590e1a with delete_type TABLET_DATA_DELETED (replaced by tablet e0648fcd521e4db19359c743447246b7) from {username='slave'} at 127.0.0.1:57378
I20260925 20:00:22.864632 4989 tablet_replica.cc:333] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:22.864847 4989 raft_consensus.cc:2243] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:22.865065 4989 raft_consensus.cc:2272] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:22.865720 4989 ts_tablet_manager.cc:1916] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:22.869704 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet 460399668df444c5af242442af590e1a with delete_type TABLET_DATA_DELETED (replaced by tablet e0648fcd521e4db19359c743447246b7) from {username='slave'} at 127.0.0.1:54948
I20260925 20:00:22.869829 4784 tablet_service.cc:1558] Processing DeleteTablet for tablet 460399668df444c5af242442af590e1a with delete_type TABLET_DATA_DELETED (replaced by tablet e0648fcd521e4db19359c743447246b7) from {username='slave'} at 127.0.0.1:54514
I20260925 20:00:22.870018 4988 tablet_replica.cc:333] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:22.870211 4988 raft_consensus.cc:2243] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:22.870230 4990 tablet_replica.cc:333] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:22.870483 4990 raft_consensus.cc:2243] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:22.870671 4988 pending_rounds.cc:70] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482: Trying to abort 1 pending ops.
I20260925 20:00:22.870713 4990 raft_consensus.cc:2272] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:22.870831 4988 pending_rounds.cc:77] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333340866267021312 op_type: NO_OP noop_request { }
I20260925 20:00:22.871004 4988 raft_consensus.cc:2889] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 20:00:22.871129 4988 raft_consensus.cc:2272] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:22.871467 4990 ts_tablet_manager.cc:1916] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:22.871510 4185 catalog_manager.cc:6934] ReplaceTablet: tablet 460399668df444c5af242442af590e1a deleted and replaced by tablet e0648fcd521e4db19359c743447246b7
I20260925 20:00:22.871655 4185 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48108 (request call id 59) took 1222 ms. Trace:
I20260925 20:00:22.871766 4185 rpcz_store.cc:276] 0925 20:00:21.648740 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:22.848696 (+1199956us) service_pool.cc:224] Handling call
0925 20:00:22.871618 (+ 22922us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 20:00:22.850668 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 20:00:22.850817 (+ 149us) write_op.cc:432] Acquiring schema lock in shared mode
0925 20:00:22.850824 (+ 7us) write_op.cc:435] Acquired schema lock
0925 20:00:22.850826 (+ 2us) tablet.cc:662] Decoding operations
0925 20:00:22.850976 (+ 150us) write_op.cc:620] Acquiring the partition lock for write op
0925 20:00:22.850991 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0925 20:00:22.850997 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0925 20:00:22.851084 (+ 87us) tablet.cc:701] Row locks acquired
0925 20:00:22.851086 (+ 2us) write_op.cc:260] PREPARE: finished
0925 20:00:22.851124 (+ 38us) write_op.cc:270] Start()
0925 20:00:22.851162 (+ 38us) write_op.cc:276] Timestamp: P: 1790366422851070 usec, L: 0
0925 20:00:22.851166 (+ 4us) op_driver.cc:348] REPLICATION: starting
0925 20:00:22.851384 (+ 218us) log.cc:844] Serialized 661 byte log entry
0925 20:00:22.859739 (+ 8355us) op_driver.cc:464] REPLICATION: finished
0925 20:00:22.859903 (+ 164us) write_op.cc:301] APPLY: starting
0925 20:00:22.859922 (+ 19us) tablet.cc:1394] starting BulkCheckPresence
0925 20:00:22.859979 (+ 57us) tablet.cc:1397] finished BulkCheckPresence
0925 20:00:22.860005 (+ 26us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 20:00:22.860113 (+ 108us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 20:00:22.860177 (+ 64us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 20:00:22.860184 (+ 7us) write_op.cc:312] APPLY: finished
0925 20:00:22.860267 (+ 83us) log.cc:844] Serialized 28 byte log entry
0925 20:00:22.860370 (+ 103us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:22.860393 (+ 23us) write_op.cc:454] Released schema lock
0925 20:00:22.860400 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0925 20:00:22.860483 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:22.860487 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":112,"apply.run_cpu_time_us":569,"apply.run_wall_time_us":568,"num_ops":2,"prepare.queue_time_us":717,"prepare.run_cpu_time_us":845,"prepare.run_wall_time_us":846,"raft.queue_time_us":421,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":241,"replication_time_us":8537}]]}
I20260925 20:00:22.872483 4988 ts_tablet_manager.cc:1916] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:22.879670 4990 ts_tablet_manager.cc:1929] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 20:00:22.879820 4990 log.cc:1199] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/460399668df444c5af242442af590e1a
I20260925 20:00:22.880666 4990 ts_tablet_manager.cc:1950] T 460399668df444c5af242442af590e1a P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:22.883478 4176 catalog_manager.cc:5063] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395): tablet 460399668df444c5af242442af590e1a (table default.table_533f83fef17a455883e73ee1c68b252a [id=25f6e1ade43c4519bed519037ee8f761]) successfully deleted
I20260925 20:00:22.886914 4647 tablet_service.cc:1511] Processing CreateTablet for tablet e0648fcd521e4db19359c743447246b7 (DEFAULT_TABLE table=default.table_533f83fef17a455883e73ee1c68b252a [id=25f6e1ade43c4519bed519037ee8f761]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.887416 4647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0648fcd521e4db19359c743447246b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.889199 4989 ts_tablet_manager.cc:1929] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 20:00:22.890286 4988 ts_tablet_manager.cc:1929] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:22.890525 4989 log.cc:1199] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/460399668df444c5af242442af590e1a
I20260925 20:00:22.890995 4989 ts_tablet_manager.cc:1950] T 460399668df444c5af242442af590e1a P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:22.891184 4988 log.cc:1199] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/460399668df444c5af242442af590e1a
I20260925 20:00:22.891641 4988 ts_tablet_manager.cc:1950] T 460399668df444c5af242442af590e1a P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:22.892728 4189 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7] alter complete (version 1)
I20260925 20:00:22.893623 4189 master_service.cc:960] ReplaceTablet: received request to replace tablet dff315b3b8e64ed0838b6241eeb442d3 from {username='slave'} at 127.0.0.1:48108
I20260925 20:00:22.894902 4174 catalog_manager.cc:5063] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): tablet 460399668df444c5af242442af590e1a (table default.table_533f83fef17a455883e73ee1c68b252a [id=25f6e1ade43c4519bed519037ee8f761]) successfully deleted
I20260925 20:00:22.896399 4505 tablet_service.cc:1511] Processing CreateTablet for tablet e0648fcd521e4db19359c743447246b7 (DEFAULT_TABLE table=default.table_533f83fef17a455883e73ee1c68b252a [id=25f6e1ade43c4519bed519037ee8f761]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.896808 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0648fcd521e4db19359c743447246b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.900671 4176 catalog_manager.cc:5063] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): tablet 460399668df444c5af242442af590e1a (table default.table_533f83fef17a455883e73ee1c68b252a [id=25f6e1ade43c4519bed519037ee8f761]) successfully deleted
I20260925 20:00:22.901279 4784 tablet_service.cc:1511] Processing CreateTablet for tablet e0648fcd521e4db19359c743447246b7 (DEFAULT_TABLE table=default.table_533f83fef17a455883e73ee1c68b252a [id=25f6e1ade43c4519bed519037ee8f761]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.901880 4784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0648fcd521e4db19359c743447246b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.910930 4932 tablet_bootstrap.cc:492] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:22.912875 4932 tablet_bootstrap.cc:654] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.913048 4929 tablet_bootstrap.cc:492] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:22.915314 4929 tablet_bootstrap.cc:654] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.919456 5010 raft_consensus.cc:493] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:22.919655 5010 raft_consensus.cc:515] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.920495 4940 raft_consensus.cc:493] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:22.920709 4940 raft_consensus.cc:515] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.920910 4525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6f526af054da1973d202e14ce22ae" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:22.921106 4525 raft_consensus.cc:2468] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 0.
I20260925 20:00:22.921428 4940 leader_election.cc:290] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:22.922525 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4b5baba575d4538b807a69eff0d90dd" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" is_pre_election: true
I20260925 20:00:22.922788 4805 raft_consensus.cc:2468] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:22.923202 4667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4b5baba575d4538b807a69eff0d90dd" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:22.923401 4667 raft_consensus.cc:2468] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:22.923780 4603 leader_election.cc:304] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:22.924156 4944 raft_consensus.cc:2804] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:22.924362 4189 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Sending DeleteTablet for 3 replicas of tablet dff315b3b8e64ed0838b6241eeb442d3
I20260925 20:00:22.924403 4944 raft_consensus.cc:493] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:22.924541 4944 raft_consensus.cc:3060] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.924656 4455 leader_election.cc:304] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:22.925083 5010 leader_election.cc:290] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:22.925381 4646 tablet_service.cc:1558] Processing DeleteTablet for tablet dff315b3b8e64ed0838b6241eeb442d3 with delete_type TABLET_DATA_DELETED (replaced by tablet 988fd177aaf3418d89c3eb5756f989ba) from {username='slave'} at 127.0.0.1:57378
I20260925 20:00:22.925670 4989 tablet_replica.cc:333] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:22.925691 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6f526af054da1973d202e14ce22ae" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" is_pre_election: true
I20260925 20:00:22.925894 4989 raft_consensus.cc:2243] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:22.925915 4805 raft_consensus.cc:2468] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 0.
I20260925 20:00:22.926330 4989 pending_rounds.cc:70] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0: Trying to abort 1 pending ops.
I20260925 20:00:22.926514 4989 pending_rounds.cc:77] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333340866352328704 op_type: NO_OP noop_request { }
I20260925 20:00:22.926651 4989 raft_consensus.cc:2889] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 20:00:22.926779 4989 raft_consensus.cc:2272] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:22.926888 4940 raft_consensus.cc:2804] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:22.926985 4940 raft_consensus.cc:493] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:22.927085 4940 raft_consensus.cc:3060] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.927945 4989 ts_tablet_manager.cc:1916] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:22.927865 4944 raft_consensus.cc:515] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.928635 4944 leader_election.cc:290] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 election: Requested vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:22.930189 4506 tablet_service.cc:1558] Processing DeleteTablet for tablet dff315b3b8e64ed0838b6241eeb442d3 with delete_type TABLET_DATA_DELETED (replaced by tablet 988fd177aaf3418d89c3eb5756f989ba) from {username='slave'} at 127.0.0.1:54948
I20260925 20:00:22.930932 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6f526af054da1973d202e14ce22ae" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
I20260925 20:00:22.931170 4805 raft_consensus.cc:3060] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.934186 4805 raft_consensus.cc:2468] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 1.
I20260925 20:00:22.934907 4603 leader_election.cc:304] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:22.935562 4944 raft_consensus.cc:2804] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:22.935451 4185 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_127c3d62fd1c4394bcd55bc6fc18bc5d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:22.935801 4944 raft_consensus.cc:697] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [term 1 LEADER]: Becoming Leader. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Running, Role: LEADER
I20260925 20:00:22.936115 4944 consensus_queue.cc:237] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.936503 4784 tablet_service.cc:1558] Processing DeleteTablet for tablet dff315b3b8e64ed0838b6241eeb442d3 with delete_type TABLET_DATA_DELETED (replaced by tablet 988fd177aaf3418d89c3eb5756f989ba) from {username='slave'} at 127.0.0.1:54514
I20260925 20:00:22.936854 4990 tablet_replica.cc:333] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:22.937146 4990 raft_consensus.cc:2243] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:22.937384 4990 raft_consensus.cc:2272] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:22.938095 4990 ts_tablet_manager.cc:1916] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:22.938148 4933 tablet_bootstrap.cc:492] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:22.938517 4189 catalog_manager.cc:6934] ReplaceTablet: tablet dff315b3b8e64ed0838b6241eeb442d3 deleted and replaced by tablet 988fd177aaf3418d89c3eb5756f989ba
I20260925 20:00:22.939123 4525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f6f526af054da1973d202e14ce22ae" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:22.939352 4525 raft_consensus.cc:3060] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.939343 4189 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48108 (request call id 60) took 1287 ms. Trace:
I20260925 20:00:22.939447 4189 rpcz_store.cc:276] 0925 20:00:21.651473 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:22.893551 (+1242078us) service_pool.cc:224] Handling call
0925 20:00:22.939276 (+ 45725us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 20:00:22.895615 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 20:00:22.895751 (+ 136us) write_op.cc:432] Acquiring schema lock in shared mode
0925 20:00:22.895758 (+ 7us) write_op.cc:435] Acquired schema lock
0925 20:00:22.895761 (+ 3us) tablet.cc:662] Decoding operations
0925 20:00:22.895885 (+ 124us) write_op.cc:620] Acquiring the partition lock for write op
0925 20:00:22.895900 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0925 20:00:22.895905 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0925 20:00:22.896018 (+ 113us) tablet.cc:701] Row locks acquired
0925 20:00:22.896026 (+ 8us) write_op.cc:260] PREPARE: finished
0925 20:00:22.896065 (+ 39us) write_op.cc:270] Start()
0925 20:00:22.896101 (+ 36us) write_op.cc:276] Timestamp: P: 1790366422896049 usec, L: 0
0925 20:00:22.896105 (+ 4us) op_driver.cc:348] REPLICATION: starting
0925 20:00:22.896326 (+ 221us) log.cc:844] Serialized 661 byte log entry
0925 20:00:22.917832 (+ 21506us) op_driver.cc:464] REPLICATION: finished
0925 20:00:22.918604 (+ 772us) write_op.cc:301] APPLY: starting
0925 20:00:22.918624 (+ 20us) tablet.cc:1394] starting BulkCheckPresence
0925 20:00:22.918673 (+ 49us) tablet.cc:1397] finished BulkCheckPresence
0925 20:00:22.918712 (+ 39us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 20:00:22.918825 (+ 113us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 20:00:22.918884 (+ 59us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 20:00:22.918891 (+ 7us) write_op.cc:312] APPLY: finished
0925 20:00:22.918988 (+ 97us) log.cc:844] Serialized 28 byte log entry
0925 20:00:22.919079 (+ 91us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:22.919106 (+ 27us) write_op.cc:454] Released schema lock
0925 20:00:22.919114 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0925 20:00:22.919194 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:22.919197 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":698,"apply.run_cpu_time_us":587,"apply.run_wall_time_us":588,"num_ops":2,"prepare.queue_time_us":66,"prepare.run_cpu_time_us":845,"prepare.run_wall_time_us":910,"raft.queue_time_us":46,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":21690}]]}
I20260925 20:00:22.940462 4933 tablet_bootstrap.cc:654] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:22.942992 4525 raft_consensus.cc:2468] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 1.
I20260925 20:00:22.944533 4940 raft_consensus.cc:515] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.945580 4785 tablet_service.cc:1511] Processing CreateTablet for tablet 988fd177aaf3418d89c3eb5756f989ba (DEFAULT_TABLE table=default.table_4ec42ee954664a658b3aad36206c53aa [id=c47fb6987d0343418ed55e663b34ab74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.946058 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 988fd177aaf3418d89c3eb5756f989ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.947499 4990 ts_tablet_manager.cc:1929] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 20:00:22.947680 4990 log.cc:1199] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/dff315b3b8e64ed0838b6241eeb442d3
I20260925 20:00:22.947870 4189 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_799678007be3498da115944e990a1785"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:22.948114 4990 ts_tablet_manager.cc:1950] T dff315b3b8e64ed0838b6241eeb442d3 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:22.948738 4647 tablet_service.cc:1511] Processing CreateTablet for tablet 988fd177aaf3418d89c3eb5756f989ba (DEFAULT_TABLE table=default.table_4ec42ee954664a658b3aad36206c53aa [id=c47fb6987d0343418ed55e663b34ab74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.949249 4647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 988fd177aaf3418d89c3eb5756f989ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.949996 4176 catalog_manager.cc:5063] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395): tablet dff315b3b8e64ed0838b6241eeb442d3 (table default.table_4ec42ee954664a658b3aad36206c53aa [id=c47fb6987d0343418ed55e663b34ab74]) successfully deleted
I20260925 20:00:22.950547 4187 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_9f19364a66794406806691ab3a701b72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:22.950687 4933 tablet_bootstrap.cc:492] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:22.950886 4933 ts_tablet_manager.cc:1403] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.007s
I20260925 20:00:22.952085 4933 raft_consensus.cc:359] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.952360 4933 raft_consensus.cc:385] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:22.952518 4933 raft_consensus.cc:740] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:22.952884 4933 consensus_queue.cc:260] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.953893 4933 ts_tablet_manager.cc:1434] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260925 20:00:22.957087 4940 leader_election.cc:290] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Requested vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:22.952772 4988 tablet_replica.cc:333] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:22.957581 4988 raft_consensus.cc:2243] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:22.957854 4988 raft_consensus.cc:2272] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:22.958568 4988 ts_tablet_manager.cc:1916] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:22.964562 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4b5baba575d4538b807a69eff0d90dd" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
I20260925 20:00:22.964931 4805 raft_consensus.cc:3060] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.968461 4805 raft_consensus.cc:2468] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
I20260925 20:00:22.969328 4457 leader_election.cc:304] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:22.970016 4940 raft_consensus.cc:2804] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:22.970705 4932 tablet_bootstrap.cc:492] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:22.970866 4932 ts_tablet_manager.cc:1403] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.060s user 0.004s sys 0.003s
I20260925 20:00:22.972079 4932 raft_consensus.cc:359] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.972313 4932 raft_consensus.cc:385] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:22.972406 4932 raft_consensus.cc:740] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:22.972685 4932 consensus_queue.cc:260] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.972105 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 988fd177aaf3418d89c3eb5756f989ba (DEFAULT_TABLE table=default.table_4ec42ee954664a658b3aad36206c53aa [id=c47fb6987d0343418ed55e663b34ab74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:22.973451 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 988fd177aaf3418d89c3eb5756f989ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:22.987850 4940 raft_consensus.cc:697] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 LEADER]: Becoming Leader. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Running, Role: LEADER
I20260925 20:00:22.988313 4940 consensus_queue.cc:237] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:22.991663 4932 ts_tablet_manager.cc:1434] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.021s user 0.000s sys 0.003s
I20260925 20:00:22.985400 4667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4b5baba575d4538b807a69eff0d90dd" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:22.996039 4936 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 20:00:22.996622 4667 raft_consensus.cc:3060] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:22.998576 4936 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 20:00:22.999001 4936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 20:00:23.002677 4933 tablet_bootstrap.cc:492] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:23.005049 4932 tablet_bootstrap.cc:492] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:23.006057 4667 raft_consensus.cc:2468] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
I20260925 20:00:23.006758 4932 tablet_bootstrap.cc:654] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:23.007028 4933 tablet_bootstrap.cc:654] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:23.011891 4989 ts_tablet_manager.cc:1929] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:23.012259 4929 tablet_bootstrap.cc:492] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:23.012475 4929 ts_tablet_manager.cc:1403] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.100s user 0.031s sys 0.037s
I20260925 20:00:23.013281 4933 tablet_bootstrap.cc:492] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:23.013412 4989 log.cc:1199] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/dff315b3b8e64ed0838b6241eeb442d3
I20260925 20:00:23.013460 4929 raft_consensus.cc:359] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.013697 4929 raft_consensus.cc:385] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:23.013474 4933 ts_tablet_manager.cc:1403] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.002s
I20260925 20:00:23.013808 4929 raft_consensus.cc:740] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:23.014113 4929 consensus_queue.cc:260] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.015338 4933 raft_consensus.cc:359] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.015645 4933 raft_consensus.cc:385] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:23.015780 4933 raft_consensus.cc:740] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:23.016158 4933 consensus_queue.cc:260] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.016489 4932 tablet_bootstrap.cc:492] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:23.016667 4932 ts_tablet_manager.cc:1403] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260925 20:00:23.017870 4933 ts_tablet_manager.cc:1434] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260925 20:00:23.018577 4989 ts_tablet_manager.cc:1950] T dff315b3b8e64ed0838b6241eeb442d3 P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:23.018836 4932 raft_consensus.cc:359] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.019064 4932 raft_consensus.cc:385] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:23.019155 4932 raft_consensus.cc:740] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:23.019397 4929 ts_tablet_manager.cc:1434] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260925 20:00:23.019399 4932 consensus_queue.cc:260] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.020334 4932 ts_tablet_manager.cc:1434] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 20:00:23.022125 4929 tablet_bootstrap.cc:492] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:23.023438 4174 catalog_manager.cc:5063] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): tablet dff315b3b8e64ed0838b6241eeb442d3 (table default.table_4ec42ee954664a658b3aad36206c53aa [id=c47fb6987d0343418ed55e663b34ab74]) successfully deleted
I20260925 20:00:23.023942 4988 ts_tablet_manager.cc:1929] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 20:00:23.024124 4988 log.cc:1199] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/dff315b3b8e64ed0838b6241eeb442d3
I20260925 20:00:23.025117 4929 tablet_bootstrap.cc:654] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:23.033521 4988 ts_tablet_manager.cc:1950] T dff315b3b8e64ed0838b6241eeb442d3 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:23.034586 4929 tablet_bootstrap.cc:492] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:23.034782 4929 ts_tablet_manager.cc:1403] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.009s
I20260925 20:00:23.035606 4929 raft_consensus.cc:359] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.035853 4929 raft_consensus.cc:385] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:23.035949 4929 raft_consensus.cc:740] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:23.036255 4929 consensus_queue.cc:260] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.037310 4176 catalog_manager.cc:5063] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): tablet dff315b3b8e64ed0838b6241eeb442d3 (table default.table_4ec42ee954664a658b3aad36206c53aa [id=c47fb6987d0343418ed55e663b34ab74]) successfully deleted
I20260925 20:00:23.038429 4929 ts_tablet_manager.cc:1434] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260925 20:00:23.076898 4931 consensus_queue.cc:1048] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:23.103513 4970 raft_consensus.cc:493] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:23.103797 4970 raft_consensus.cc:515] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.104874 4970 leader_election.cc:290] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:23.106155 4525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0648fcd521e4db19359c743447246b7" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:23.106503 4525 raft_consensus.cc:2468] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:23.107141 4741 leader_election.cc:304] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:23.109282 4666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0648fcd521e4db19359c743447246b7" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:23.109587 4666 raft_consensus.cc:2468] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:23.115680 4970 consensus_queue.cc:1048] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:23.116390 4970 raft_consensus.cc:2804] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:23.116549 4970 raft_consensus.cc:493] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:23.116674 4970 raft_consensus.cc:3060] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:23.135195 4970 raft_consensus.cc:515] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.137319 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0648fcd521e4db19359c743447246b7" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:23.137624 4526 raft_consensus.cc:3060] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:23.141678 4970 leader_election.cc:290] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Requested vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:23.144678 4181 catalog_manager.cc:3541] renamed table default.table_533f83fef17a455883e73ee1c68b252a [id=25f6e1ade43c4519bed519037ee8f761] in HMS: new name default.table_703616ef44ba43f9afb9c4df73eb372c
I20260925 20:00:23.142367 4526 raft_consensus.cc:2468] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:23.146204 4741 leader_election.cc:304] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:23.146672 4970 raft_consensus.cc:2804] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:23.146888 4970 raft_consensus.cc:697] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Becoming Leader. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Running, Role: LEADER
I20260925 20:00:23.147281 4970 consensus_queue.cc:237] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.156002 4666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0648fcd521e4db19359c743447246b7" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:23.156384 4666 raft_consensus.cc:3060] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:23.160019 4666 raft_consensus.cc:2468] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:23.205119 4944 consensus_queue.cc:1048] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:23.218915 4184 catalog_manager.cc:2213] created HMS catalog entry for table [id=f6fa165ad5e7471589cb63403548ab2c]
I20260925 20:00:23.231796 5010 consensus_queue.cc:1048] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:23.239110 4184 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48108 (request call id 51) took 1860 ms. Trace:
I20260925 20:00:23.239281 4184 rpcz_store.cc:276] 0925 20:00:21.378485 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:21.690408 (+311923us) service_pool.cc:224] Handling call
0925 20:00:22.820782 (+1130374us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 20:00:22.821327 (+ 545us) catalog_manager.cc:2186] Created new table and tablet info
0925 20:00:23.218851 (+397524us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 20:00:23.238633 (+ 19782us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 20:00:23.238803 (+ 170us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 20:00:23.238987 (+ 184us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 20:00:23.220064 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 20:00:23.220247 (+ 183us) write_op.cc:432] Acquiring schema lock in shared mode
0925 20:00:23.220257 (+ 10us) write_op.cc:435] Acquired schema lock
0925 20:00:23.220262 (+ 5us) tablet.cc:662] Decoding operations
0925 20:00:23.220422 (+ 160us) write_op.cc:620] Acquiring the partition lock for write op
0925 20:00:23.220444 (+ 22us) write_op.cc:641] Partition lock acquired for write op
0925 20:00:23.220470 (+ 26us) tablet.cc:685] Acquiring locks for 2 operations
0925 20:00:23.220647 (+ 177us) tablet.cc:701] Row locks acquired
0925 20:00:23.220651 (+ 4us) write_op.cc:260] PREPARE: finished
0925 20:00:23.220714 (+ 63us) write_op.cc:270] Start()
0925 20:00:23.220766 (+ 52us) write_op.cc:276] Timestamp: P: 1790366423220665 usec, L: 0
0925 20:00:23.220773 (+ 7us) op_driver.cc:348] REPLICATION: starting
0925 20:00:23.221045 (+ 272us) log.cc:844] Serialized 432 byte log entry
0925 20:00:23.233972 (+ 12927us) op_driver.cc:464] REPLICATION: finished
0925 20:00:23.234170 (+ 198us) write_op.cc:301] APPLY: starting
0925 20:00:23.234196 (+ 26us) tablet.cc:1394] starting BulkCheckPresence
0925 20:00:23.234262 (+ 66us) tablet.cc:1397] finished BulkCheckPresence
0925 20:00:23.234314 (+ 52us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 20:00:23.234475 (+ 161us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 20:00:23.234527 (+ 52us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 20:00:23.234540 (+ 13us) write_op.cc:312] APPLY: finished
0925 20:00:23.234662 (+ 122us) log.cc:844] Serialized 28 byte log entry
0925 20:00:23.234757 (+ 95us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:23.234793 (+ 36us) write_op.cc:454] Released schema lock
0925 20:00:23.234804 (+ 11us) write_op.cc:341] FINISH: Updating metrics
0925 20:00:23.234917 (+ 113us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:23.234924 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":322929,"Hive Metastore.run_cpu_time_us":337,"Hive Metastore.run_wall_time_us":74036,"child_traces":[["op",{"apply.queue_time_us":124,"apply.run_cpu_time_us":733,"apply.run_wall_time_us":732,"num_ops":2,"prepare.queue_time_us":170,"prepare.run_cpu_time_us":1128,"prepare.run_wall_time_us":1129,"raft.queue_time_us":4238,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":355,"replication_time_us":13142}]]}
I20260925 20:00:23.256982 5097 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.199.253:41897 every 8 connection(s)
I20260925 20:00:23.258064 4936 rpc_server.cc:307] RPC server started. Bound to: 127.3.199.253:41897
I20260925 20:00:23.261454 4936 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
I20260925 20:00:23.263010 4647 tablet_service.cc:1511] Processing CreateTablet for tablet 44f42691903d4e2bb3f24ecb75a3f86e (DEFAULT_TABLE table=default.table_08807903f9884f2a9707d4066e7f2f52 [id=f6fa165ad5e7471589cb63403548ab2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:23.263264 4785 tablet_service.cc:1511] Processing CreateTablet for tablet 44f42691903d4e2bb3f24ecb75a3f86e (DEFAULT_TABLE table=default.table_08807903f9884f2a9707d4066e7f2f52 [id=f6fa165ad5e7471589cb63403548ab2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:23.263509 4647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44f42691903d4e2bb3f24ecb75a3f86e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:23.263248 4184 catalog_manager.cc:5838] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194). New cstate: current_term: 1 leader_uuid: "46a6f571dfa749b7905d8458d4c929b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: UNKNOWN } } }
I20260925 20:00:23.263687 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44f42691903d4e2bb3f24ecb75a3f86e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:23.263788 4184 catalog_manager.cc:5938] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647) does not have the latest schema for tablet 88f6f526af054da1973d202e14ce22ae (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]). Expected version 1 got 0
I20260925 20:00:23.265833 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 44f42691903d4e2bb3f24ecb75a3f86e (DEFAULT_TABLE table=default.table_08807903f9884f2a9707d4066e7f2f52 [id=f6fa165ad5e7471589cb63403548ab2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:23.266357 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44f42691903d4e2bb3f24ecb75a3f86e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:23.268950 4933 tablet_bootstrap.cc:492] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:23.269727 3871 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu as pid 4936
I20260925 20:00:23.270779 4933 tablet_bootstrap.cc:654] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:23.272058 4929 tablet_bootstrap.cc:492] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:23.274060 4929 tablet_bootstrap.cc:654] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:23.276553 4932 tablet_bootstrap.cc:492] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:23.285130 4932 tablet_bootstrap.cc:654] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:23.291949 5098 sys_catalog.cc:263] Verifying existing consensus state
I20260925 20:00:23.298276 5098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Bootstrap starting.
I20260925 20:00:23.302124 4184 catalog_manager.cc:5838] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 reported cstate change: term changed from 0 to 1, leader changed from <none> to 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193). New cstate: current_term: 1 leader_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: UNKNOWN } } }
I20260925 20:00:23.302628 4184 catalog_manager.cc:5938] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741) does not have the latest schema for tablet f4b5baba575d4538b807a69eff0d90dd (table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]). Expected version 1 got 0
I20260925 20:00:23.309922 4183 catalog_manager.cc:2213] created HMS catalog entry for table [id=b25b053e66e341d4bcb5a1c4dee4983d]
I20260925 20:00:23.311502 5098 log.cc:826] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Log is configured to *not* fsync() on all Append() calls
I20260925 20:00:23.318192 4932 tablet_bootstrap.cc:492] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:23.318408 4932 ts_tablet_manager.cc:1403] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.042s user 0.004s sys 0.000s
I20260925 20:00:23.319485 4932 raft_consensus.cc:359] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
I20260925 20:00:23.319690 4932 raft_consensus.cc:385] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:23.319784 4932 raft_consensus.cc:740] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:23.320094 4932 consensus_queue.cc:260] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
I20260925 20:00:23.320933 4932 ts_tablet_manager.cc:1434] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260925 20:00:23.330178 4805 raft_consensus.cc:1275] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Refusing update from remote peer 46a6f571dfa749b7905d8458d4c929b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 20:00:23.330287 4525 raft_consensus.cc:1275] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Refusing update from remote peer 46a6f571dfa749b7905d8458d4c929b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 20:00:23.331103 4944 consensus_queue.cc:1048] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:23.331704 4944 consensus_queue.cc:1048] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:23.339296 5098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 20:00:23.340159 5098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Bootstrap complete.
I20260925 20:00:23.353452 5098 raft_consensus.cc:359] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:23.354633 5098 raft_consensus.cc:740] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: da29ee550f82456fa7c5ce7ea916909f, State: Initialized, Role: FOLLOWER
I20260925 20:00:23.355780 5098 consensus_queue.cc:260] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:23.372551 5110 tablet.cc:1751] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 20:00:23.394922 4944 consensus_queue.cc:1048] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:23.396539 4852 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:23.396857 4852 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:23.403152 5111 tablet.cc:1751] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 20:00:23.405966 4929 tablet_bootstrap.cc:492] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:23.406167 4929 ts_tablet_manager.cc:1403] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.134s user 0.018s sys 0.026s
I20260925 20:00:23.407095 4929 raft_consensus.cc:359] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
I20260925 20:00:23.407379 4929 raft_consensus.cc:385] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:23.407503 4929 raft_consensus.cc:740] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:23.407868 4929 consensus_queue.cc:260] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
I20260925 20:00:23.408807 4929 ts_tablet_manager.cc:1434] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260925 20:00:23.430073 4852 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:23.442648 4183 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48108 (request call id 52) took 1948 ms. Trace:
I20260925 20:00:23.442831 4183 rpcz_store.cc:276] 0925 20:00:21.494039 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:21.930683 (+436644us) service_pool.cc:224] Handling call
0925 20:00:22.820891 (+890208us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 20:00:22.821389 (+ 498us) catalog_manager.cc:2186] Created new table and tablet info
0925 20:00:23.309885 (+488496us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 20:00:23.442292 (+132407us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 20:00:23.442434 (+ 142us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 20:00:23.442570 (+ 136us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 20:00:23.312092 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 20:00:23.312314 (+ 222us) write_op.cc:432] Acquiring schema lock in shared mode
0925 20:00:23.312324 (+ 10us) write_op.cc:435] Acquired schema lock
0925 20:00:23.312328 (+ 4us) tablet.cc:662] Decoding operations
0925 20:00:23.312553 (+ 225us) write_op.cc:620] Acquiring the partition lock for write op
0925 20:00:23.312568 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0925 20:00:23.312578 (+ 10us) tablet.cc:685] Acquiring locks for 2 operations
0925 20:00:23.312705 (+ 127us) tablet.cc:701] Row locks acquired
0925 20:00:23.312708 (+ 3us) write_op.cc:260] PREPARE: finished
0925 20:00:23.312754 (+ 46us) write_op.cc:270] Start()
0925 20:00:23.312839 (+ 85us) write_op.cc:276] Timestamp: P: 1790366423312700 usec, L: 0
0925 20:00:23.312845 (+ 6us) op_driver.cc:348] REPLICATION: starting
0925 20:00:23.417764 (+104919us) log.cc:844] Serialized 432 byte log entry
0925 20:00:23.418233 (+ 469us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x616000043654. stack: 00007fc4f63c05c6 00007fc4f63c0380 00007fc4f53c834d 00000000009a4cb4 00000000009a4c7d 00000000009a464a 00007fc50248a1b2 00007fc5038a7fa5 00007fc5038a6ac9 00007fc5038aeef0 00007fc5038aecfd 0000000000aaef9c 00007fc4f6406aef 00007fc4f6409deb 00007fc4f6409c68 0000000000aaef9c
0925 20:00:23.441142 (+ 22909us) op_driver.cc:464] REPLICATION: finished
0925 20:00:23.441539 (+ 397us) write_op.cc:301] APPLY: starting
0925 20:00:23.441560 (+ 21us) tablet.cc:1394] starting BulkCheckPresence
0925 20:00:23.441612 (+ 52us) tablet.cc:1397] finished BulkCheckPresence
0925 20:00:23.441614 (+ 2us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 20:00:23.441768 (+ 154us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 20:00:23.441810 (+ 42us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 20:00:23.441818 (+ 8us) write_op.cc:312] APPLY: finished
0925 20:00:23.441922 (+ 104us) log.cc:844] Serialized 28 byte log entry
0925 20:00:23.442033 (+ 111us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:23.442061 (+ 28us) write_op.cc:454] Released schema lock
0925 20:00:23.442069 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0925 20:00:23.442156 (+ 87us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:23.442162 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":397015,"Hive Metastore.run_cpu_time_us":326,"Hive Metastore.run_wall_time_us":91203,"child_traces":[["op",{"apply.queue_time_us":307,"apply.run_cpu_time_us":619,"apply.run_wall_time_us":618,"num_ops":2,"prepare.queue_time_us":204,"prepare.run_cpu_time_us":2951,"prepare.run_wall_time_us":106366,"raft.queue_time_us":713,"raft.run_cpu_time_us":403,"raft.run_wall_time_us":403,"replication_time_us":128248,"spinlock_wait_cycles":240471424}]]}
I20260925 20:00:23.444868 4184 catalog_manager.cc:5838] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e reported cstate change: term changed from 0 to 1, leader changed from <none> to dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195). New cstate: current_term: 1 leader_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: HEALTHY } } }
I20260925 20:00:23.450788 4933 tablet_bootstrap.cc:492] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:23.450982 4933 ts_tablet_manager.cc:1403] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.182s user 0.036s sys 0.035s
I20260925 20:00:23.451014 4852 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:23.451890 4933 raft_consensus.cc:359] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
I20260925 20:00:23.452102 4933 raft_consensus.cc:385] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:23.452198 4933 raft_consensus.cc:740] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:23.452507 4933 consensus_queue.cc:260] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
I20260925 20:00:23.453584 4933 ts_tablet_manager.cc:1434] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 20:00:23.457612 4573 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:23.457418 5098 sys_catalog.cc:565] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 20:00:23.454312 5112 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:23.459884 5112 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:23.462060 4666 raft_consensus.cc:1275] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Refusing update from remote peer 145cdcf7953e4c0d9686b1c6a86f9482: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 20:00:23.462586 4805 raft_consensus.cc:1275] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Refusing update from remote peer 145cdcf7953e4c0d9686b1c6a86f9482: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 20:00:23.463451 4940 consensus_queue.cc:1048] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:23.464397 4714 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:23.464300 4940 consensus_queue.cc:1048] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:23.469837 4714 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:23.480005 5014 mvcc.cc:204] Tried to move back new op lower bound from 7333340870492901376 to 7333340868564684800. Current Snapshot: MvccSnapshot[applied={T|T < 7333340870492901376}]
I20260925 20:00:23.487105 5110 tablet.cc:1751] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 20:00:23.487319 5013 mvcc.cc:204] Tried to move back new op lower bound from 7333340870492901376 to 7333340868564684800. Current Snapshot: MvccSnapshot[applied={T|T < 7333340870492901376}]
I20260925 20:00:23.496192 5111 tablet.cc:1751] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 20:00:23.521435 4183 catalog_manager.cc:5938] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741) does not have the latest schema for tablet 88f6f526af054da1973d202e14ce22ae (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]). Expected version 1 got 0
I20260925 20:00:23.525452 4647 tablet_service.cc:1511] Processing CreateTablet for tablet 8d0bc4cceef042389ad99a3515aa0f43 (DEFAULT_TABLE table=default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:23.526165 4647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d0bc4cceef042389ad99a3515aa0f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:23.533977 4785 tablet_service.cc:1511] Processing CreateTablet for tablet 8d0bc4cceef042389ad99a3515aa0f43 (DEFAULT_TABLE table=default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:23.534475 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d0bc4cceef042389ad99a3515aa0f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:23.537158 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 8d0bc4cceef042389ad99a3515aa0f43 (DEFAULT_TABLE table=default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:23.537628 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d0bc4cceef042389ad99a3515aa0f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:23.542670 4184 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812] alter complete (version 1)
I20260925 20:00:23.545367 4184 catalog_manager.cc:5938] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395) does not have the latest schema for tablet f4b5baba575d4538b807a69eff0d90dd (table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]). Expected version 1 got 0
I20260925 20:00:23.547734 4183 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_b6819354070b414c8a47d9f19a011a5c" } modify_external_catalogs: true
I20260925 20:00:23.548022 4183 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_b6819354070b414c8a47d9f19a011a5c" } modify_external_catalogs: true
I20260925 20:00:23.548477 4184 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_5fdc6db4e1ee47848136cec254f09a72" } modify_external_catalogs: true
I20260925 20:00:23.548678 4184 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_5fdc6db4e1ee47848136cec254f09a72" } modify_external_catalogs: true
I20260925 20:00:23.552479 4852 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:23.559904 4932 tablet_bootstrap.cc:492] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:23.562242 4932 tablet_bootstrap.cc:654] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:23.567248 4944 consensus_queue.cc:1048] T 2ddb9fbd0aeb4c4a985508ac67c93227 P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 20:00:23.568523 4180 catalog_manager.cc:2213] created HMS catalog entry for table [id=7ead82097c49448f85b1dbef027be066]
I20260925 20:00:23.579396 4180 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48108 (request call id 54) took 2042 ms. Trace:
I20260925 20:00:23.579587 4180 rpcz_store.cc:276] 0925 20:00:21.537192 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:22.187879 (+650687us) service_pool.cc:224] Handling call
0925 20:00:22.821926 (+634047us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 20:00:22.822354 (+ 428us) catalog_manager.cc:2186] Created new table and tablet info
0925 20:00:23.568464 (+746110us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 20:00:23.578943 (+ 10479us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 20:00:23.579124 (+ 181us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 20:00:23.579291 (+ 167us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 20:00:23.569523 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 20:00:23.569658 (+ 135us) write_op.cc:432] Acquiring schema lock in shared mode
0925 20:00:23.569666 (+ 8us) write_op.cc:435] Acquired schema lock
0925 20:00:23.569668 (+ 2us) tablet.cc:662] Decoding operations
0925 20:00:23.569820 (+ 152us) write_op.cc:620] Acquiring the partition lock for write op
0925 20:00:23.569836 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0925 20:00:23.569841 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0925 20:00:23.569937 (+ 96us) tablet.cc:701] Row locks acquired
0925 20:00:23.569940 (+ 3us) write_op.cc:260] PREPARE: finished
0925 20:00:23.569986 (+ 46us) write_op.cc:270] Start()
0925 20:00:23.570026 (+ 40us) write_op.cc:276] Timestamp: P: 1790366423569954 usec, L: 0
0925 20:00:23.570030 (+ 4us) op_driver.cc:348] REPLICATION: starting
0925 20:00:23.570275 (+ 245us) log.cc:844] Serialized 432 byte log entry
0925 20:00:23.577637 (+ 7362us) op_driver.cc:464] REPLICATION: finished
0925 20:00:23.577881 (+ 244us) write_op.cc:301] APPLY: starting
0925 20:00:23.577909 (+ 28us) tablet.cc:1394] starting BulkCheckPresence
0925 20:00:23.578000 (+ 91us) tablet.cc:1397] finished BulkCheckPresence
0925 20:00:23.578088 (+ 88us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 20:00:23.578261 (+ 173us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 20:00:23.578318 (+ 57us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 20:00:23.578332 (+ 14us) write_op.cc:312] APPLY: finished
0925 20:00:23.578465 (+ 133us) log.cc:844] Serialized 28 byte log entry
0925 20:00:23.578558 (+ 93us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:23.578587 (+ 29us) write_op.cc:454] Released schema lock
0925 20:00:23.578612 (+ 25us) write_op.cc:341] FINISH: Updating metrics
0925 20:00:23.578749 (+ 137us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:23.578754 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":488691,"Hive Metastore.run_cpu_time_us":363,"Hive Metastore.run_wall_time_us":257007,"child_traces":[["op",{"apply.queue_time_us":150,"apply.run_cpu_time_us":828,"apply.run_wall_time_us":828,"mutex_wait_us":356,"num_ops":2,"prepare.queue_time_us":77,"prepare.run_cpu_time_us":917,"prepare.run_wall_time_us":1264,"raft.queue_time_us":453,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":291,"replication_time_us":7550}]]}
I20260925 20:00:23.557912 4929 tablet_bootstrap.cc:492] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:23.609601 4929 tablet_bootstrap.cc:654] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:23.611074 4944 raft_consensus.cc:493] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:23.611249 4944 raft_consensus.cc:515] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
I20260925 20:00:23.612457 4804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f42691903d4e2bb3f24ecb75a3f86e" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" is_pre_election: true
I20260925 20:00:23.612692 4804 raft_consensus.cc:2468] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 0.
I20260925 20:00:23.613147 4525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f42691903d4e2bb3f24ecb75a3f86e" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:23.613433 4525 raft_consensus.cc:2468] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 0.
I20260925 20:00:23.614024 4603 leader_election.cc:304] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:23.614434 5010 raft_consensus.cc:2804] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:23.614539 5010 raft_consensus.cc:493] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:23.614658 5010 raft_consensus.cc:3060] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:23.615964 5065 raft_consensus.cc:3060] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 2 FOLLOWER]: Advancing to term 3
I20260925 20:00:23.617700 5010 raft_consensus.cc:515] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
I20260925 20:00:23.618875 5010 leader_election.cc:290] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 election: Requested vote from peers dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395), 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:23.619961 4525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f42691903d4e2bb3f24ecb75a3f86e" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:23.620208 4525 raft_consensus.cc:3060] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:23.624356 4944 leader_election.cc:290] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395), 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:23.637171 4646 tablet_service.cc:1511] Processing CreateTablet for tablet 59a14370409146a1847c5b76755e3feb (DEFAULT_TABLE table=default.table_c78f9cc1d7e74b51b29858c7d3248fb8 [id=7ead82097c49448f85b1dbef027be066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:23.637770 4646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59a14370409146a1847c5b76755e3feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:23.637791 4929 tablet_bootstrap.cc:492] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:23.638018 4929 ts_tablet_manager.cc:1403] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.080s user 0.006s sys 0.004s
I20260925 20:00:23.639004 4929 raft_consensus.cc:359] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.639633 4929 raft_consensus.cc:385] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:23.639739 4929 raft_consensus.cc:740] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:23.640105 4929 consensus_queue.cc:260] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.640978 4929 ts_tablet_manager.cc:1434] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 20:00:23.644603 4180 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22] alter complete (version 1)
I20260925 20:00:23.650347 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 59a14370409146a1847c5b76755e3feb (DEFAULT_TABLE table=default.table_c78f9cc1d7e74b51b29858c7d3248fb8 [id=7ead82097c49448f85b1dbef027be066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:23.650993 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59a14370409146a1847c5b76755e3feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:23.657200 4785 tablet_service.cc:1511] Processing CreateTablet for tablet 59a14370409146a1847c5b76755e3feb (DEFAULT_TABLE table=default.table_c78f9cc1d7e74b51b29858c7d3248fb8 [id=7ead82097c49448f85b1dbef027be066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:23.657694 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59a14370409146a1847c5b76755e3feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:23.660053 4933 tablet_bootstrap.cc:492] T 59a14370409146a1847c5b76755e3feb P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:23.662452 4933 tablet_bootstrap.cc:654] T 59a14370409146a1847c5b76755e3feb P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:23.666498 4929 tablet_bootstrap.cc:492] T 59a14370409146a1847c5b76755e3feb P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:23.668533 4929 tablet_bootstrap.cc:654] T 59a14370409146a1847c5b76755e3feb P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:23.676011 4929 tablet_bootstrap.cc:492] T 59a14370409146a1847c5b76755e3feb P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:23.676240 4929 ts_tablet_manager.cc:1403] T 59a14370409146a1847c5b76755e3feb P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260925 20:00:23.677438 4929 raft_consensus.cc:359] T 59a14370409146a1847c5b76755e3feb P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.677706 4929 raft_consensus.cc:385] T 59a14370409146a1847c5b76755e3feb P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:23.677815 4929 raft_consensus.cc:740] T 59a14370409146a1847c5b76755e3feb P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:23.678117 4929 consensus_queue.cc:260] T 59a14370409146a1847c5b76755e3feb P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.679059 4929 ts_tablet_manager.cc:1434] T 59a14370409146a1847c5b76755e3feb P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 20:00:23.681074 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f42691903d4e2bb3f24ecb75a3f86e" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
I20260925 20:00:23.681345 4805 raft_consensus.cc:3060] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:23.688866 4852 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
W20260925 20:00:23.690999 5110 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.203s user 0.000s sys 0.001s
I20260925 20:00:23.691279 4933 tablet_bootstrap.cc:492] T 59a14370409146a1847c5b76755e3feb P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:23.691493 4933 ts_tablet_manager.cc:1403] T 59a14370409146a1847c5b76755e3feb P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.032s user 0.010s sys 0.000s
I20260925 20:00:23.692910 4933 raft_consensus.cc:359] T 59a14370409146a1847c5b76755e3feb P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.693161 4933 raft_consensus.cc:385] T 59a14370409146a1847c5b76755e3feb P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:23.693266 4933 raft_consensus.cc:740] T 59a14370409146a1847c5b76755e3feb P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:23.693558 4933 consensus_queue.cc:260] T 59a14370409146a1847c5b76755e3feb P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.694202 4933 ts_tablet_manager.cc:1434] T 59a14370409146a1847c5b76755e3feb P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260925 20:00:23.694568 4933 tablet_bootstrap.cc:492] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:23.696893 5120 tablet.cc:1751] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 20:00:23.697479 4933 tablet_bootstrap.cc:654] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:23.701035 5116 tablet.cc:1751] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 20:00:23.702699 4525 raft_consensus.cc:2468] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 1.
I20260925 20:00:23.703366 4603 leader_election.cc:304] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:23.703814 5113 raft_consensus.cc:2804] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:23.704059 5113 raft_consensus.cc:697] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [term 1 LEADER]: Becoming Leader. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Running, Role: LEADER
I20260925 20:00:23.704396 5113 consensus_queue.cc:237] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
I20260925 20:00:23.708657 4932 tablet_bootstrap.cc:492] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:23.708962 4932 ts_tablet_manager.cc:1403] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.149s user 0.012s sys 0.044s
I20260925 20:00:23.709981 4932 raft_consensus.cc:359] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.710212 4932 raft_consensus.cc:385] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:23.710290 4932 raft_consensus.cc:740] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:23.710562 4932 consensus_queue.cc:260] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.711145 4932 ts_tablet_manager.cc:1434] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260925 20:00:23.711481 4932 tablet_bootstrap.cc:492] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:23.713907 4932 tablet_bootstrap.cc:654] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:23.721740 4933 tablet_bootstrap.cc:492] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:23.721937 4933 ts_tablet_manager.cc:1403] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.005s
I20260925 20:00:23.722838 4933 raft_consensus.cc:359] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.723073 4933 raft_consensus.cc:385] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:23.723165 4933 raft_consensus.cc:740] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:23.723465 4933 consensus_queue.cc:260] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.724044 4933 ts_tablet_manager.cc:1434] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 20:00:23.736004 4932 tablet_bootstrap.cc:492] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:23.738821 4932 ts_tablet_manager.cc:1403] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.014s
I20260925 20:00:23.741520 4932 raft_consensus.cc:359] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.741762 4932 raft_consensus.cc:385] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:23.741871 4932 raft_consensus.cc:740] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:23.742247 4932 consensus_queue.cc:260] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.745396 4932 ts_tablet_manager.cc:1434] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260925 20:00:23.764698 4970 raft_consensus.cc:493] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:23.764961 4970 raft_consensus.cc:515] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.765425 4805 raft_consensus.cc:2468] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 1.
I20260925 20:00:23.767134 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988fd177aaf3418d89c3eb5756f989ba" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:23.767439 4526 raft_consensus.cc:2468] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:23.768132 4666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988fd177aaf3418d89c3eb5756f989ba" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:23.768409 4666 raft_consensus.cc:2468] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:23.769182 4741 leader_election.cc:304] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:23.765802 4970 leader_election.cc:290] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:23.770041 4970 raft_consensus.cc:2804] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:23.770155 4970 raft_consensus.cc:493] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:23.770287 4970 raft_consensus.cc:3060] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:23.773259 4970 raft_consensus.cc:515] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.776517 4666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988fd177aaf3418d89c3eb5756f989ba" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:23.776795 4666 raft_consensus.cc:3060] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:23.779943 4666 raft_consensus.cc:2468] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:23.781239 4739 leader_election.cc:304] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:23.781929 4970 leader_election.cc:290] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Requested vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:23.782133 4970 raft_consensus.cc:2804] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:23.782274 4970 raft_consensus.cc:697] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Becoming Leader. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Running, Role: LEADER
I20260925 20:00:23.782552 4970 consensus_queue.cc:237] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.795063 5113 consensus_queue.cc:1048] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:23.796473 4852 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:23.817353 4180 catalog_manager.cc:5838] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194). New cstate: current_term: 1 leader_uuid: "46a6f571dfa749b7905d8458d4c929b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: UNKNOWN } } }
W20260925 20:00:23.818629 4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.199.193:34741, user_credentials={real_user=slave}} blocked reactor thread for 49478.9us
I20260925 20:00:23.824460 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988fd177aaf3418d89c3eb5756f989ba" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:23.824834 4526 raft_consensus.cc:3060] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:23.831496 4526 raft_consensus.cc:2468] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:23.838667 5113 consensus_queue.cc:1048] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:23.843497 4852 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:23.875604 4852 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:23.889331 5038 raft_consensus.cc:493] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:23.889546 5038 raft_consensus.cc:515] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.889961 4852 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:23.890352 5038 leader_election.cc:290] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:23.893566 4666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a14370409146a1847c5b76755e3feb" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:23.893889 4666 raft_consensus.cc:2468] T 59a14370409146a1847c5b76755e3feb P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:23.894510 4455 leader_election.cc:304] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:23.896576 5038 raft_consensus.cc:2804] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:23.896727 5038 raft_consensus.cc:493] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:23.896834 5038 raft_consensus.cc:3060] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:23.900108 5038 raft_consensus.cc:515] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.900697 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a14370409146a1847c5b76755e3feb" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" is_pre_election: true
I20260925 20:00:23.901356 4805 raft_consensus.cc:2468] T 59a14370409146a1847c5b76755e3feb P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:23.902644 4666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a14370409146a1847c5b76755e3feb" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:23.902894 4666 raft_consensus.cc:3060] T 59a14370409146a1847c5b76755e3feb P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:23.905355 4180 catalog_manager.cc:5838] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e reported cstate change: term changed from 0 to 1, leader changed from <none> to dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195). New cstate: current_term: 1 leader_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: HEALTHY } } }
I20260925 20:00:23.906296 4666 raft_consensus.cc:2468] T 59a14370409146a1847c5b76755e3feb P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
I20260925 20:00:23.907665 4455 leader_election.cc:304] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:23.908165 5037 raft_consensus.cc:2804] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:23.908341 5037 raft_consensus.cc:697] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 LEADER]: Becoming Leader. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Running, Role: LEADER
I20260925 20:00:23.908718 5037 consensus_queue.cc:237] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:23.913743 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a14370409146a1847c5b76755e3feb" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
I20260925 20:00:23.914422 4805 raft_consensus.cc:3060] T 59a14370409146a1847c5b76755e3feb P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:23.914640 5038 leader_election.cc:290] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Requested vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:23.918957 4805 raft_consensus.cc:2468] T 59a14370409146a1847c5b76755e3feb P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
I20260925 20:00:23.940953 4180 catalog_manager.cc:5838] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 reported cstate change: term changed from 0 to 1, leader changed from <none> to 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193). New cstate: current_term: 1 leader_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: UNKNOWN } } }
I20260925 20:00:23.945957 5112 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 169b6de20560440b99cc08af549ce477. Latest consensus state: current_term: 3 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:23.946203 5112 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:23.946537 5145 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:23.946767 5145 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:23.949873 5104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:23.950153 5104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:23.962307 4186 catalog_manager.cc:3541] renamed table default.table_56db49c71dc44c579bb1fa4eebb56bbb [id=86f1c8a16cc14ef9b72c95060313b3ad] in HMS: new name default.table_098b200909f842c88a38f11192f4dc25
I20260925 20:00:23.963577 4180 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_7722a249d10f4616ba5b5bb2fc149c18" } modify_external_catalogs: true
I20260925 20:00:23.963975 4180 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_7722a249d10f4616ba5b5bb2fc149c18" } modify_external_catalogs: true
I20260925 20:00:23.995075 4970 consensus_queue.cc:1048] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:24.009413 4852 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:24.011765 5132 consensus_queue.cc:1048] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:24.040408 5135 mvcc.cc:204] Tried to move back new op lower bound from 7333340871018450944 to 7333340860441071616. Current Snapshot: MvccSnapshot[applied={T|T < 7333340871018450944 or (T in {7333340871018450944})}]
I20260925 20:00:24.048660 5160 catalog_manager.cc:1308] Loaded cluster ID: 7efb83138ca94380bf2d8f0f2f91c381
I20260925 20:00:24.048780 5160 catalog_manager.cc:1601] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: loading cluster ID for follower catalog manager: success
I20260925 20:00:24.059805 5160 catalog_manager.cc:1623] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: acquiring CA information for follower catalog manager: success
I20260925 20:00:24.063263 5054 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" instance_seqno: 1790366419525613) as {username='slave'} at 127.3.199.194:37307; Asking this server to re-register.
I20260925 20:00:24.065132 4714 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:24.072448 5054 ts_manager.cc:194] Registered new tserver with Master: 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:24.077473 5160 catalog_manager.cc:1651] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 20:00:24.334779 5132 raft_consensus.cc:493] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:24.335012 5132 raft_consensus.cc:515] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:24.335742 5132 leader_election.cc:290] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:24.336864 4666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0bc4cceef042389ad99a3515aa0f43" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:24.337102 4666 raft_consensus.cc:2468] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:24.337085 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0bc4cceef042389ad99a3515aa0f43" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:24.337284 4526 raft_consensus.cc:2468] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:24.337661 4739 leader_election.cc:304] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:24.338697 5132 raft_consensus.cc:2804] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:24.338804 5132 raft_consensus.cc:493] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:24.338913 5132 raft_consensus.cc:3060] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:24.341974 5132 raft_consensus.cc:515] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:24.343405 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0bc4cceef042389ad99a3515aa0f43" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:24.343639 4526 raft_consensus.cc:3060] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:24.344853 5132 leader_election.cc:290] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Requested vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:24.347221 4526 raft_consensus.cc:2468] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:24.347808 4741 leader_election.cc:304] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:24.348788 4666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d0bc4cceef042389ad99a3515aa0f43" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:24.349036 4666 raft_consensus.cc:3060] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:24.352370 4666 raft_consensus.cc:2468] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:24.353404 5132 raft_consensus.cc:2804] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:24.353626 5132 raft_consensus.cc:697] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Becoming Leader. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Running, Role: LEADER
I20260925 20:00:24.353899 5132 consensus_queue.cc:237] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:24.355955 4852 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:24.356226 4852 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:24.384732 5054 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" instance_seqno: 1790366420065491) as {username='slave'} at 127.3.199.195:43745; Asking this server to re-register.
I20260925 20:00:24.385957 4852 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:24.388621 5054 ts_manager.cc:194] Registered new tserver with Master: dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:24.464720 5169 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 169b6de20560440b99cc08af549ce477)
I20260925 20:00:24.465034 5169 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:24.466437 4199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" candidate_term: 4 candidate_status { last_received { term: 3 index: 63 } } ignore_live_leader: false dest_uuid: "169b6de20560440b99cc08af549ce477" is_pre_election: true
I20260925 20:00:24.472790 5170 raft_consensus.cc:493] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 169b6de20560440b99cc08af549ce477)
I20260925 20:00:24.479650 5169 leader_election.cc:290] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897)
I20260925 20:00:24.480089 5170 raft_consensus.cc:515] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:24.483251 4199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da29ee550f82456fa7c5ce7ea916909f" candidate_term: 4 candidate_status { last_received { term: 3 index: 63 } } ignore_live_leader: false dest_uuid: "169b6de20560440b99cc08af549ce477" is_pre_election: true
I20260925 20:00:24.484369 5170 leader_election.cc:290] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:24.498414 5065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" candidate_term: 4 candidate_status { last_received { term: 3 index: 63 } } ignore_live_leader: false dest_uuid: "da29ee550f82456fa7c5ce7ea916909f" is_pre_election: true
I20260925 20:00:24.498879 5065 raft_consensus.cc:2468] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f43a6a14ef547f2aeb22ce2b334383e in term 3.
I20260925 20:00:24.499709 4315 leader_election.cc:304] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f; no voters: 169b6de20560440b99cc08af549ce477
I20260925 20:00:24.500130 5169 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260925 20:00:24.500331 5169 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 169b6de20560440b99cc08af549ce477)
I20260925 20:00:24.500485 5169 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 3 FOLLOWER]: Advancing to term 4
I20260925 20:00:24.504204 5169 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:24.505640 4199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" candidate_term: 4 candidate_status { last_received { term: 3 index: 63 } } ignore_live_leader: false dest_uuid: "169b6de20560440b99cc08af549ce477"
I20260925 20:00:24.505831 5169 leader_election.cc:290] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [CANDIDATE]: Term 4 election: Requested vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897)
I20260925 20:00:24.507009 5065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" candidate_term: 4 candidate_status { last_received { term: 3 index: 63 } } ignore_live_leader: false dest_uuid: "da29ee550f82456fa7c5ce7ea916909f"
I20260925 20:00:24.507288 5065 raft_consensus.cc:3060] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 3 FOLLOWER]: Advancing to term 4
I20260925 20:00:24.511279 5065 raft_consensus.cc:2468] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f43a6a14ef547f2aeb22ce2b334383e in term 4.
I20260925 20:00:24.513200 4315 leader_election.cc:304] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f; no voters: 169b6de20560440b99cc08af549ce477
I20260925 20:00:24.513696 5169 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 4 FOLLOWER]: Leader election won for term 4
I20260925 20:00:24.524724 4573 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:24.529599 5054 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" instance_seqno: 1790366418849966) as {username='slave'} at 127.3.199.193:46149; Asking this server to re-register.
I20260925 20:00:24.531121 4573 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:24.533176 5054 ts_manager.cc:194] Registered new tserver with Master: 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:24.534302 4338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da29ee550f82456fa7c5ce7ea916909f" candidate_term: 4 candidate_status { last_received { term: 3 index: 63 } } ignore_live_leader: false dest_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" is_pre_election: true
I20260925 20:00:24.542196 5169 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 4 LEADER]: Becoming Leader. State: Replica: 7f43a6a14ef547f2aeb22ce2b334383e, State: Running, Role: LEADER
I20260925 20:00:24.542836 5169 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 63, Committed index: 63, Last appended: 3.63, Last appended by leader: 63, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:24.545702 5032 leader_election.cc:304] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da29ee550f82456fa7c5ce7ea916909f; no voters: 169b6de20560440b99cc08af549ce477, 7f43a6a14ef547f2aeb22ce2b334383e
I20260925 20:00:24.546669 4339 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [term 4 LEADER]: Rejecting Update request from peer 169b6de20560440b99cc08af549ce477 for earlier term 3. Current term is 4. Ops: []
I20260925 20:00:24.546687 5170 raft_consensus.cc:2749] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260925 20:00:24.547984 5175 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 }, Status: INVALID_TERM, Last received: 3.63, Next index: 64, Last known committed idx: 63, Time since last communication: 0.000s
I20260925 20:00:24.555742 5174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f43a6a14ef547f2aeb22ce2b334383e. Latest consensus state: current_term: 4 leader_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:24.558533 5174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:24.562660 5175 consensus_queue.cc:1243] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [LEADER]: Peer 7f43a6a14ef547f2aeb22ce2b334383e log is divergent from this leader: its last log entry 4.64 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 63
I20260925 20:00:24.566102 5176 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 3 LEADER]: Stepping down as leader of term 3
I20260925 20:00:24.566419 5176 raft_consensus.cc:740] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 169b6de20560440b99cc08af549ce477, State: Running, Role: LEADER
I20260925 20:00:24.566888 5176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 63, Committed index: 63, Last appended: 3.63, Last appended by leader: 63, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
W20260925 20:00:24.567312 4175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 20:00:24.567361 5176 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 3 FOLLOWER]: Advancing to term 4
I20260925 20:00:24.573396 5177 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260925 20:00:24.576620 5177 catalog_manager.cc:692] Loaded metadata for table default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330]
I20260925 20:00:24.577184 5177 catalog_manager.cc:692] Loaded metadata for table default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf]
I20260925 20:00:24.577620 5177 catalog_manager.cc:692] Loaded metadata for table default.table_533f83fef17a455883e73ee1c68b252a [id=25f6e1ade43c4519bed519037ee8f761]
I20260925 20:00:24.578049 5177 catalog_manager.cc:692] Loaded metadata for table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]
I20260925 20:00:24.578460 5177 catalog_manager.cc:692] Loaded metadata for table default.table_c78f9cc1d7e74b51b29858c7d3248fb8 [id=7ead82097c49448f85b1dbef027be066]
I20260925 20:00:24.579041 5177 catalog_manager.cc:692] Loaded metadata for table default.table_56db49c71dc44c579bb1fa4eebb56bbb [id=86f1c8a16cc14ef9b72c95060313b3ad]
I20260925 20:00:24.579627 5177 catalog_manager.cc:692] Loaded metadata for table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]
I20260925 20:00:24.596042 5177 catalog_manager.cc:692] Loaded metadata for table default.table_7722a249d10f4616ba5b5bb2fc149c18 [id=a8f568d7faab469ba7ea8a2a38bfd638]
I20260925 20:00:24.596630 5177 catalog_manager.cc:692] Loaded metadata for table default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d]
I20260925 20:00:24.597082 5177 catalog_manager.cc:692] Loaded metadata for table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]
I20260925 20:00:24.597483 5177 catalog_manager.cc:692] Loaded metadata for table default.table_4ec42ee954664a658b3aad36206c53aa [id=c47fb6987d0343418ed55e663b34ab74]
I20260925 20:00:24.597906 5177 catalog_manager.cc:692] Loaded metadata for table default.table_08807903f9884f2a9707d4066e7f2f52 [id=f6fa165ad5e7471589cb63403548ab2c]
I20260925 20:00:24.600322 5177 tablet_loader.cc:96] loaded metadata for tablet 0cd27940acbb43ef9d75591871c00481 (table default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330])
I20260925 20:00:24.601075 5177 tablet_loader.cc:96] loaded metadata for tablet 2ddb9fbd0aeb4c4a985508ac67c93227 (table default.table_7722a249d10f4616ba5b5bb2fc149c18 [id=a8f568d7faab469ba7ea8a2a38bfd638])
I20260925 20:00:24.601456 5177 tablet_loader.cc:96] loaded metadata for tablet 44f42691903d4e2bb3f24ecb75a3f86e (table default.table_08807903f9884f2a9707d4066e7f2f52 [id=f6fa165ad5e7471589cb63403548ab2c])
I20260925 20:00:24.602023 5177 tablet_loader.cc:96] loaded metadata for tablet 59a14370409146a1847c5b76755e3feb (table default.table_c78f9cc1d7e74b51b29858c7d3248fb8 [id=7ead82097c49448f85b1dbef027be066])
I20260925 20:00:24.602573 5177 tablet_loader.cc:96] loaded metadata for tablet 635a8a28ca8444dd8162e29a405810d6 (table default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf])
I20260925 20:00:24.602957 5177 tablet_loader.cc:96] loaded metadata for tablet 88f6f526af054da1973d202e14ce22ae (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812])
I20260925 20:00:24.603297 5177 tablet_loader.cc:96] loaded metadata for tablet 8d0bc4cceef042389ad99a3515aa0f43 (table default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d])
I20260925 20:00:24.603631 5177 tablet_loader.cc:96] loaded metadata for tablet 983d375d3fda4e63ab93b6cde278e187 (table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7])
I20260925 20:00:24.607513 5177 tablet_loader.cc:96] loaded metadata for tablet 988fd177aaf3418d89c3eb5756f989ba (table default.table_4ec42ee954664a658b3aad36206c53aa [id=c47fb6987d0343418ed55e663b34ab74])
I20260925 20:00:24.608438 5177 tablet_loader.cc:96] loaded metadata for tablet e0648fcd521e4db19359c743447246b7 (table default.table_533f83fef17a455883e73ee1c68b252a [id=25f6e1ade43c4519bed519037ee8f761])
I20260925 20:00:24.608799 5177 tablet_loader.cc:96] loaded metadata for tablet ee843d55757347ef9942dffd917c0ff8 (table default.table_56db49c71dc44c579bb1fa4eebb56bbb [id=86f1c8a16cc14ef9b72c95060313b3ad])
I20260925 20:00:24.609167 5177 tablet_loader.cc:96] loaded metadata for tablet f4b5baba575d4538b807a69eff0d90dd (table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22])
I20260925 20:00:24.609701 5177 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260925 20:00:24.611158 5177 catalog_manager.cc:1308] Loaded cluster ID: 7efb83138ca94380bf2d8f0f2f91c381
I20260925 20:00:24.611248 5177 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260925 20:00:24.612986 5177 catalog_manager.cc:1553] Loading token signing keys...
I20260925 20:00:24.614205 5177 catalog_manager.cc:6230] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e: Loaded TSK: 0
I20260925 20:00:24.615296 5177 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260925 20:00:24.616532 5177 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260925 20:00:24.617823 5177 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: 13
I20260925 20:00:24.707681 4712 heartbeater.cc:499] Master 127.3.199.252:37957 was elected leader, sending a full tablet report...
I20260925 20:00:24.724781 5037 consensus_queue.cc:1048] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:24.739135 4182 catalog_manager.cc:2770] deleted HMS catalog entry for table default.table_4ec42ee954664a658b3aad36206c53aa [id=c47fb6987d0343418ed55e663b34ab74]
I20260925 20:00:24.750924 5168 consensus_queue.cc:1048] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:24.752435 4850 heartbeater.cc:499] Master 127.3.199.252:37957 was elected leader, sending a full tablet report...
I20260925 20:00:24.774746 4328 catalog_manager.cc:5838] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e reported cstate change: term changed from 0 to 1, leader changed from <none> to dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195). New cstate: current_term: 1 leader_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: HEALTHY } } }
I20260925 20:00:24.778739 5167 consensus_queue.cc:1048] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:24.794528 5179 consensus_queue.cc:1048] T 59a14370409146a1847c5b76755e3feb P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:24.807096 5132 consensus_queue.cc:1048] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:24.857702 5132 consensus_queue.cc:1048] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:24.884506 5065 raft_consensus.cc:1275] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 4 FOLLOWER]: Refusing update from remote peer 7f43a6a14ef547f2aeb22ce2b334383e: Log matching property violated. Preceding OpId in replica: term: 3 index: 63. Preceding OpId from leader: term: 4 index: 65. (index mismatch)
I20260925 20:00:24.885028 4188 catalog_manager.cc:2213] created HMS catalog entry for table [id=44287e882a4040dcb79d824c08808dd4]
W20260925 20:00:24.886231 4188 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica 169b6de20560440b99cc08af549ce477 is not leader of this config: current role FOLLOWER
I20260925 20:00:24.887367 5174 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 63, Time since last communication: 0.000s
I20260925 20:00:24.891692 4199 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 4 FOLLOWER]: Refusing update from remote peer 7f43a6a14ef547f2aeb22ce2b334383e: Log matching property violated. Preceding OpId in replica: term: 3 index: 63. Preceding OpId from leader: term: 4 index: 65. (index mismatch)
I20260925 20:00:24.892954 5174 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [LEADER]: Connected to new peer: Peer: permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 63, Time since last communication: 0.000s
I20260925 20:00:24.900684 5176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f43a6a14ef547f2aeb22ce2b334383e. Latest consensus state: current_term: 4 leader_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:24.900996 5176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 20:00:24.913966 4403 catalog_manager.cc:3978] Illegal state: an error occurred while updating the sys-catalog: replica 169b6de20560440b99cc08af549ce477 is not leader of this config: current role FOLLOWER
W20260925 20:00:24.914422 4403 hms_notification_log_listener.cc:330] Failed to handle Hive Metastore notification: 14 ALTER_TABLE default.table_703616ef44ba43f9afb9c4df73eb372c: Illegal state: an error occurred while updating the sys-catalog: replica 169b6de20560440b99cc08af549ce477 is not leader of this config: current role FOLLOWER
I20260925 20:00:24.916800 5170 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f43a6a14ef547f2aeb22ce2b334383e. Latest consensus state: current_term: 4 leader_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:24.917083 5170 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:24.920252 5176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:24.920506 5176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 20:00:24.921136 4403 hms_notification_log_listener.cc:143] Hive Metastore notification log listener poll failed: Service unavailable: lost leadership while handling Hive Metastore notification log events: an error occurred while updating the sys-catalog: replica 169b6de20560440b99cc08af549ce477 is not leader of this config: current role FOLLOWER
I20260925 20:00:24.921720 4185 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48108 (request call id 61) took 3209 ms. Trace:
I20260925 20:00:24.921811 4185 rpcz_store.cc:276] 0925 20:00:21.711940 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:22.935180 (+1223240us) service_pool.cc:224] Handling call
0925 20:00:24.921659 (+1986479us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260925 20:00:24.922964 4185 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.199.193:58133 (request call id 27) took 917 ms (client timeout 1000 ms). Trace:
W20260925 20:00:24.923067 4185 rpcz_store.cc:269] 0925 20:00:24.005913 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.922100 (+916187us) service_pool.cc:224] Handling call
0925 20:00:24.922870 (+ 770us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260925 20:00:24.924093 4185 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.199.195:41065 (request call id 30) took 913 ms (client timeout 1000 ms). Trace:
W20260925 20:00:24.924187 4185 rpcz_store.cc:269] 0925 20:00:24.010616 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.923300 (+912684us) service_pool.cc:224] Handling call
0925 20:00:24.924037 (+ 737us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260925 20:00:24.930145 4862 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.199.254:41289)
I20260925 20:00:24.932842 4571 heartbeater.cc:499] Master 127.3.199.252:37957 was elected leader, sending a full tablet report...
I20260925 20:00:24.944672 4326 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52966:
name: "default.Table_127c3d62fd1c4394bcd55bc6fc18bc5d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:24.948836 5169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:24.949141 5169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: This master's current role is: LEADER
W20260925 20:00:24.952700 4185 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.3.199.194:58391 (request call id 27) took 888 ms (client timeout 1000 ms). Trace:
W20260925 20:00:24.925161 4187 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_9f19364a66794406806691ab3a701b72 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 20:00:24.925845 4181 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48108 (request call id 53) took 3413 ms. Trace:
I20260925 20:00:24.956046 4181 rpcz_store.cc:276] 0925 20:00:21.512322 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:22.178926 (+666604us) service_pool.cc:224] Handling call
0925 20:00:22.820233 (+641307us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 20:00:24.925283 (+2105050us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 20:00:24.925545 (+ 262us) catalog_manager.cc:3764] Apply alter schema
0925 20:00:24.925812 (+ 267us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":6527,"Hive Metastore.run_cpu_time_us":745,"Hive Metastore.run_wall_time_us":317363}
I20260925 20:00:24.958454 5174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:24.958716 5174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:24.958774 4181 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48108 (request call id 67) took 2735 ms. Trace:
I20260925 20:00:24.958913 4181 rpcz_store.cc:276] 0925 20:00:22.223048 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.958422 (+2735374us) service_pool.cc:224] Handling call
0925 20:00:24.958740 (+ 318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:24.959389 4181 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48108 (request call id 68) took 2634 ms. Trace:
I20260925 20:00:24.959497 4181 rpcz_store.cc:276] 0925 20:00:22.325435 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.959094 (+2633659us) service_pool.cc:224] Handling call
0925 20:00:24.959356 (+ 262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:24.959957 4181 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48108 (request call id 69) took 2585 ms. Trace:
I20260925 20:00:24.960060 4181 rpcz_store.cc:276] 0925 20:00:22.374936 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.959668 (+2584732us) service_pool.cc:224] Handling call
0925 20:00:24.959924 (+ 256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:24.926662 4186 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48108 (request call id 50) took 3589 ms. Trace:
I20260925 20:00:24.961018 4186 rpcz_store.cc:276] 0925 20:00:21.337036 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:21.597830 (+260794us) service_pool.cc:224] Handling call
0925 20:00:22.823543 (+1225713us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 20:00:24.926239 (+2102696us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 20:00:24.926428 (+ 189us) catalog_manager.cc:3764] Apply alter schema
0925 20:00:24.926626 (+ 198us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":744537,"Hive Metastore.run_cpu_time_us":790,"Hive Metastore.run_wall_time_us":393912}
I20260925 20:00:24.961524 4186 master_service.cc:960] ReplaceTablet: received request to replace tablet 460399668df444c5af242442af590e1a from {username='slave'} at 127.0.0.1:48108
I20260925 20:00:24.961880 4186 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48108 (request call id 71) took 2213 ms. Trace:
I20260925 20:00:24.961966 4186 rpcz_store.cc:276] 0925 20:00:22.748382 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.961465 (+2213083us) service_pool.cc:224] Handling call
0925 20:00:24.961815 (+ 350us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:24.962508 4186 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:48108 (request call id 72) took 2136 ms. Trace:
I20260925 20:00:24.962600 4186 rpcz_store.cc:276] 0925 20:00:22.826222 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.962193 (+2135971us) service_pool.cc:224] Handling call
0925 20:00:24.962455 (+ 262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:24.962909 4186 master_service.cc:960] ReplaceTablet: received request to replace tablet 88f6f526af054da1973d202e14ce22ae from {username='slave'} at 127.0.0.1:48108
I20260925 20:00:24.963238 4186 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48108 (request call id 73) took 2129 ms. Trace:
I20260925 20:00:24.963332 4186 rpcz_store.cc:276] 0925 20:00:22.833411 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.962829 (+2129418us) service_pool.cc:224] Handling call
0925 20:00:24.963207 (+ 378us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:24.963789 4186 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48108 (request call id 74) took 1937 ms. Trace:
I20260925 20:00:24.963904 4186 rpcz_store.cc:276] 0925 20:00:23.026711 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.963482 (+1936771us) service_pool.cc:224] Handling call
0925 20:00:24.963751 (+ 269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:24.927191 4182 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:48108 (request call id 58) took 3332 ms. Trace:
I20260925 20:00:24.964408 4182 rpcz_store.cc:276] 0925 20:00:21.594811 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:22.626002 (+1031191us) service_pool.cc:224] Handling call
0925 20:00:22.823336 (+197334us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 20:00:24.927159 (+2103823us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":1137852,"Hive Metastore.run_cpu_time_us":426,"Hive Metastore.run_wall_time_us":776831}
I20260925 20:00:24.964716 4182 master_service.cc:960] ReplaceTablet: received request to replace tablet 988fd177aaf3418d89c3eb5756f989ba from {username='slave'} at 127.0.0.1:48108
I20260925 20:00:24.965040 4182 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48108 (request call id 76) took 1885 ms. Trace:
I20260925 20:00:24.965121 4182 rpcz_store.cc:276] 0925 20:00:23.079906 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.964667 (+1884761us) service_pool.cc:224] Handling call
0925 20:00:24.964992 (+ 325us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:24.965328 4182 master_service.cc:960] ReplaceTablet: received request to replace tablet 635a8a28ca8444dd8162e29a405810d6 from {username='slave'} at 127.0.0.1:48108
I20260925 20:00:24.965612 4182 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48108 (request call id 77) took 1819 ms. Trace:
I20260925 20:00:24.965708 4182 rpcz_store.cc:276] 0925 20:00:23.146198 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.965265 (+1819067us) service_pool.cc:224] Handling call
0925 20:00:24.965580 (+ 315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:24.966190 4182 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48108 (request call id 78) took 1721 ms. Trace:
I20260925 20:00:24.966293 4182 rpcz_store.cc:276] 0925 20:00:23.244905 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.965892 (+1720987us) service_pool.cc:224] Handling call
0925 20:00:24.966156 (+ 264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:24.966759 4182 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48108 (request call id 79) took 1519 ms. Trace:
I20260925 20:00:24.966877 4182 rpcz_store.cc:276] 0925 20:00:23.447314 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.966437 (+1519123us) service_pool.cc:224] Handling call
0925 20:00:24.966721 (+ 284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:24.967327 4182 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48108 (request call id 80) took 1318 ms. Trace:
I20260925 20:00:24.967432 4182 rpcz_store.cc:276] 0925 20:00:23.648803 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.967047 (+1318244us) service_pool.cc:224] Handling call
0925 20:00:24.967289 (+ 242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:24.967849 4182 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48108 (request call id 81) took 1182 ms. Trace:
I20260925 20:00:24.967959 4182 rpcz_store.cc:276] 0925 20:00:23.785583 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.967583 (+1182000us) service_pool.cc:224] Handling call
0925 20:00:24.967814 (+ 231us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260925 20:00:24.924768 4189 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_799678007be3498da115944e990a1785 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 20:00:24.970995 4185 rpcz_store.cc:269] 0925 20:00:24.064201 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.951748 (+887547us) service_pool.cc:224] Handling call
0925 20:00:24.952630 (+ 882us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:24.977151 4186 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:48108 (request call id 75) took 1950 ms. Trace:
I20260925 20:00:24.977392 4186 rpcz_store.cc:276] 0925 20:00:23.026894 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.964060 (+1937166us) service_pool.cc:224] Handling call
0925 20:00:24.977093 (+ 13033us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:24.978071 4186 rpcz_store.cc:275] Call kudu.master.MasterService.GetMasterRegistration from 127.0.0.1:48082 (request call id 4) took 2119 ms. Trace:
I20260925 20:00:24.978237 4186 rpcz_store.cc:276] 0925 20:00:22.858797 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.977657 (+2118860us) service_pool.cc:224] Handling call
0925 20:00:24.978034 (+ 377us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:24.979264 4182 rpcz_store.cc:275] Call kudu.master.MasterService.GetMasterRegistration from 127.0.0.1:48060 (request call id 2) took 1970 ms. Trace:
I20260925 20:00:24.980711 4182 rpcz_store.cc:276] 0925 20:00:23.009088 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.978968 (+1969880us) service_pool.cc:224] Handling call
0925 20:00:24.979234 (+ 266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:24.983660 4181 master_service.cc:960] ReplaceTablet: received request to replace tablet dff315b3b8e64ed0838b6241eeb442d3 from {username='slave'} at 127.0.0.1:48108
I20260925 20:00:24.984880 4181 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48108 (request call id 70) took 2500 ms. Trace:
I20260925 20:00:24.997401 4181 rpcz_store.cc:276] 0925 20:00:22.484454 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:24.960224 (+2475770us) service_pool.cc:224] Handling call
0925 20:00:24.984803 (+ 24579us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 20:00:25.039347 4328 catalog_manager.cc:5938] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395) does not have the latest schema for tablet e0648fcd521e4db19359c743447246b7 (table default.table_703616ef44ba43f9afb9c4df73eb372c [id=25f6e1ade43c4519bed519037ee8f761]). Expected version 1 got 0
I20260925 20:00:25.043082 4385 catalog_manager.cc:2811] Deleting table default.table_4ec42ee954664a658b3aad36206c53aa [id=c47fb6987d0343418ed55e663b34ab74] in response to Hive Metastore notification log event 19
I20260925 20:00:25.043713 5170 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:25.043946 5170 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:25.048877 4328 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52966:
name: "default.Table_5b8386185e8e4fcd99110bd5137a578c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:25.052438 4385 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e: Sending DeleteTablet for 3 replicas of tablet 988fd177aaf3418d89c3eb5756f989ba
I20260925 20:00:25.058845 4327 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52966:
name: "default.Table_667eace0e712438bbd1211f7e3e665f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:25.071349 4329 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52966:
name: "default.Table_2ebd7ecb76b44d13a6847408f7fd7d17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 20:00:25.075855 4862 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.3.199.252:37957)
W20260925 20:00:25.076710 4862 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.3.199.252:37957)
W20260925 20:00:25.077261 4862 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.199.252:37957)
I20260925 20:00:25.079779 4325 master_service.cc:960] ReplaceTablet: received request to replace tablet 460399668df444c5af242442af590e1a from {username='slave'} at 127.0.0.1:52966
I20260925 20:00:25.083431 4325 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52966:
table { table_name: "default.Table_c860d14624744708bf4ff50a84517b59" } modify_external_catalogs: true
I20260925 20:00:25.083719 4325 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52966:
table { table_name: "default.Table_c860d14624744708bf4ff50a84517b59" } modify_external_catalogs: true
I20260925 20:00:25.091135 4324 master_service.cc:960] ReplaceTablet: received request to replace tablet 88f6f526af054da1973d202e14ce22ae from {username='slave'} at 127.0.0.1:52966
I20260925 20:00:25.095445 4323 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52966:
table { table_name: "default.Table_b74d63554f344195800ce065c2a3cddf" } new_table_name: "default.Table_1f3deb6e388f4abf83c97af376b6cf64" modify_external_catalogs: true
I20260925 20:00:25.098812 4324 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e: Sending DeleteTablet for 3 replicas of tablet 88f6f526af054da1973d202e14ce22ae
I20260925 20:00:25.099395 4324 catalog_manager.cc:6934] ReplaceTablet: tablet 88f6f526af054da1973d202e14ce22ae deleted and replaced by tablet 72c3c64108f348a5ab893e6954d72d77
W20260925 20:00:25.099440 4326 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_127c3d62fd1c4394bcd55bc6fc18bc5d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 20:00:25.100461 4324 master_service.cc:960] ReplaceTablet: received request to replace tablet 988fd177aaf3418d89c3eb5756f989ba from {username='slave'} at 127.0.0.1:52966
I20260925 20:00:25.102973 4322 master_service.cc:960] ReplaceTablet: received request to replace tablet 635a8a28ca8444dd8162e29a405810d6 from {username='slave'} at 127.0.0.1:52966
W20260925 20:00:25.106361 4328 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_5b8386185e8e4fcd99110bd5137a578c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 20:00:25.107111 4321 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52966:
name: "default.Table_6796b08b82a249d8b499df979ebeb52a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 20:00:25.107770 4327 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_667eace0e712438bbd1211f7e3e665f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 20:00:25.108434 4320 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52966:
name: "default.Table_f051ca2de9a64755b465c9b043ce4392"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:25.109336 4324 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52966:
table { table_name: "default.Table_08807903f9884f2a9707d4066e7f2f52" } new_table_name: "default.Table_6ea0804544d84afca71f1f239845be69" modify_external_catalogs: true
W20260925 20:00:25.110509 4329 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_2ebd7ecb76b44d13a6847408f7fd7d17 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 20:00:25.111500 4322 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e: Sending DeleteTablet for 3 replicas of tablet 635a8a28ca8444dd8162e29a405810d6
I20260925 20:00:25.112056 4322 catalog_manager.cc:6934] ReplaceTablet: tablet 635a8a28ca8444dd8162e29a405810d6 deleted and replaced by tablet b59f74f459c542a88e8bef5f1c51357d
I20260925 20:00:25.112489 4322 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52966:
name: "default.Table_f9d13749640c4f11913f7449b48cf6c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:25.137742 4782 tablet_service.cc:1558] Processing DeleteTablet for tablet 88f6f526af054da1973d202e14ce22ae with delete_type TABLET_DATA_DELETED (replaced by tablet 72c3c64108f348a5ab893e6954d72d77) from {username='slave'} at 127.0.0.1:54530
I20260925 20:00:25.138641 4781 tablet_service.cc:1558] Processing DeleteTablet for tablet 635a8a28ca8444dd8162e29a405810d6 with delete_type TABLET_DATA_DELETED (replaced by tablet b59f74f459c542a88e8bef5f1c51357d) from {username='slave'} at 127.0.0.1:54530
I20260925 20:00:25.138731 4780 tablet_service.cc:1511] Processing CreateTablet for tablet b59f74f459c542a88e8bef5f1c51357d (DEFAULT_TABLE table=default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.139113 4785 tablet_service.cc:1511] Processing CreateTablet for tablet 72c3c64108f348a5ab893e6954d72d77 (DEFAULT_TABLE table=default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.139441 4780 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b59f74f459c542a88e8bef5f1c51357d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:25.139658 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72c3c64108f348a5ab893e6954d72d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:25.137773 4783 tablet_service.cc:1558] Processing DeleteTablet for tablet 988fd177aaf3418d89c3eb5756f989ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 20:00:25 UTC) from {username='slave'} at 127.0.0.1:54530
I20260925 20:00:25.148887 4647 tablet_service.cc:1558] Processing DeleteTablet for tablet 988fd177aaf3418d89c3eb5756f989ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 20:00:25 UTC) from {username='slave'} at 127.0.0.1:57394
I20260925 20:00:25.160938 4646 tablet_service.cc:1558] Processing DeleteTablet for tablet 88f6f526af054da1973d202e14ce22ae with delete_type TABLET_DATA_DELETED (replaced by tablet 72c3c64108f348a5ab893e6954d72d77) from {username='slave'} at 127.0.0.1:57394
I20260925 20:00:25.165469 4646 tablet_service.cc:1511] Processing CreateTablet for tablet 72c3c64108f348a5ab893e6954d72d77 (DEFAULT_TABLE table=default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.166137 4646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72c3c64108f348a5ab893e6954d72d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:25.174216 5213 tablet_replica.cc:333] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:25.176810 4645 tablet_service.cc:1558] Processing DeleteTablet for tablet 635a8a28ca8444dd8162e29a405810d6 with delete_type TABLET_DATA_DELETED (replaced by tablet b59f74f459c542a88e8bef5f1c51357d) from {username='slave'} at 127.0.0.1:57394
I20260925 20:00:25.177265 4644 tablet_service.cc:1511] Processing CreateTablet for tablet b59f74f459c542a88e8bef5f1c51357d (DEFAULT_TABLE table=default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.177861 4644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b59f74f459c542a88e8bef5f1c51357d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:25.181538 5216 tablet_replica.cc:333] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:25.183091 5216 raft_consensus.cc:2243] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:25.183445 5216 raft_consensus.cc:2272] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:25.184450 5216 ts_tablet_manager.cc:1916] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:25.182096 5218 tablet_bootstrap.cc:492] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:25.195977 5218 tablet_bootstrap.cc:654] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:25.201241 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet 988fd177aaf3418d89c3eb5756f989ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 20:00:25 UTC) from {username='slave'} at 127.0.0.1:54960
I20260925 20:00:25.201341 4506 tablet_service.cc:1558] Processing DeleteTablet for tablet 88f6f526af054da1973d202e14ce22ae with delete_type TABLET_DATA_DELETED (replaced by tablet 72c3c64108f348a5ab893e6954d72d77) from {username='slave'} at 127.0.0.1:54960
I20260925 20:00:25.202193 4506 tablet_service.cc:1511] Processing CreateTablet for tablet 72c3c64108f348a5ab893e6954d72d77 (DEFAULT_TABLE table=default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.202782 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72c3c64108f348a5ab893e6954d72d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:25.208890 4504 tablet_service.cc:1558] Processing DeleteTablet for tablet 635a8a28ca8444dd8162e29a405810d6 with delete_type TABLET_DATA_DELETED (replaced by tablet b59f74f459c542a88e8bef5f1c51357d) from {username='slave'} at 127.0.0.1:54960
I20260925 20:00:25.209928 4504 tablet_service.cc:1511] Processing CreateTablet for tablet b59f74f459c542a88e8bef5f1c51357d (DEFAULT_TABLE table=default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.210572 4504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b59f74f459c542a88e8bef5f1c51357d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:25.233897 5225 tablet.cc:1751] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 20:00:25.236532 5223 tablet_replica.cc:333] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:25.237923 5223 raft_consensus.cc:2243] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:25.239161 5223 raft_consensus.cc:2272] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:25.240790 5218 tablet_bootstrap.cc:492] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:25.243677 5223 ts_tablet_manager.cc:1916] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:25.245304 5226 tablet.cc:1751] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 20:00:25.246124 5228 tablet_bootstrap.cc:492] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:25.181099 5217 tablet_bootstrap.cc:492] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:25.249079 5217 tablet_bootstrap.cc:654] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:25.249439 5218 ts_tablet_manager.cc:1403] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.067s user 0.006s sys 0.000s
I20260925 20:00:25.256515 5213 raft_consensus.cc:2243] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:25.266702 5230 tablet.cc:1751] T ee843d55757347ef9942dffd917c0ff8 P dd2623c3b2c3497ca1df4a676fa6807e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 20:00:25.269409 5213 raft_consensus.cc:2272] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:25.270136 5213 ts_tablet_manager.cc:1916] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:25.245736 5224 tablet.cc:1751] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 20:00:25.274330 5218 raft_consensus.cc:359] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:25.268646 5225 tablet.cc:1751] T ee843d55757347ef9942dffd917c0ff8 P 46a6f571dfa749b7905d8458d4c929b0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 20:00:25.278204 5217 tablet_bootstrap.cc:492] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:25.278455 5217 ts_tablet_manager.cc:1403] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.097s user 0.000s sys 0.004s
I20260925 20:00:25.279367 5217 raft_consensus.cc:359] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:25.279664 5217 raft_consensus.cc:385] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:25.279819 5217 raft_consensus.cc:740] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:25.280237 5217 consensus_queue.cc:260] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:25.280773 5218 raft_consensus.cc:385] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:25.280967 5217 ts_tablet_manager.cc:1434] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 20:00:25.280957 5218 raft_consensus.cc:740] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:25.281174 5218 consensus_queue.cc:260] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:25.284214 5218 ts_tablet_manager.cc:1434] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260925 20:00:25.286140 5216 ts_tablet_manager.cc:1929] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:25.289996 5216 log.cc:1199] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/988fd177aaf3418d89c3eb5756f989ba
I20260925 20:00:25.287284 5218 tablet_bootstrap.cc:492] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:25.294914 5223 ts_tablet_manager.cc:1929] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:25.295091 5223 log.cc:1199] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/988fd177aaf3418d89c3eb5756f989ba
I20260925 20:00:25.293254 5216 ts_tablet_manager.cc:1950] T 988fd177aaf3418d89c3eb5756f989ba P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:25.298233 4326 catalog_manager.cc:2213] created HMS catalog entry for table [id=45cbddfbaced49a989c1fb61a644a713]
I20260925 20:00:25.301895 5223 ts_tablet_manager.cc:1950] T 988fd177aaf3418d89c3eb5756f989ba P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:25.302090 5228 tablet_bootstrap.cc:654] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:25.303706 4316 catalog_manager.cc:5063] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): tablet 988fd177aaf3418d89c3eb5756f989ba (table default.table_4ec42ee954664a658b3aad36206c53aa [id=c47fb6987d0343418ed55e663b34ab74]) successfully deleted
I20260925 20:00:25.296859 5218 tablet_bootstrap.cc:654] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:25.304100 4314 catalog_manager.cc:5063] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): tablet 988fd177aaf3418d89c3eb5756f989ba (table default.table_4ec42ee954664a658b3aad36206c53aa [id=c47fb6987d0343418ed55e663b34ab74]) successfully deleted
I20260925 20:00:25.305346 5216 tablet_replica.cc:333] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:25.305497 5223 tablet_replica.cc:333] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:25.305610 5216 raft_consensus.cc:2243] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:25.305727 5223 raft_consensus.cc:2243] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:25.305943 5223 raft_consensus.cc:2272] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:25.306043 5216 raft_consensus.cc:2272] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:25.306610 5223 ts_tablet_manager.cc:1916] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:25.306609 5216 ts_tablet_manager.cc:1916] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:25.306747 5217 tablet_bootstrap.cc:492] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:25.308557 5217 tablet_bootstrap.cc:654] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:25.314694 5213 ts_tablet_manager.cc:1929] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:25.314857 5213 log.cc:1199] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/88f6f526af054da1973d202e14ce22ae
I20260925 20:00:25.315312 5213 ts_tablet_manager.cc:1950] T 88f6f526af054da1973d202e14ce22ae P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:25.325997 5218 tablet_bootstrap.cc:492] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:25.326207 5218 ts_tablet_manager.cc:1403] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.004s
I20260925 20:00:25.327128 5218 raft_consensus.cc:359] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.327334 5218 raft_consensus.cc:385] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:25.327409 5218 raft_consensus.cc:740] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:25.327641 5218 consensus_queue.cc:260] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.328186 5218 ts_tablet_manager.cc:1434] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260925 20:00:25.330322 5213 tablet_replica.cc:333] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:25.330729 4316 catalog_manager.cc:5063] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395): tablet 88f6f526af054da1973d202e14ce22ae (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]) successfully deleted
I20260925 20:00:25.331624 5228 tablet_bootstrap.cc:492] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:25.331784 5228 ts_tablet_manager.cc:1403] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.086s user 0.002s sys 0.005s
I20260925 20:00:25.332612 5228 raft_consensus.cc:359] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:25.332785 5228 raft_consensus.cc:385] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:25.332870 5228 raft_consensus.cc:740] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:25.333160 5228 consensus_queue.cc:260] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:25.335096 5228 ts_tablet_manager.cc:1434] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260925 20:00:25.335750 5213 raft_consensus.cc:2243] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:25.335894 5228 tablet_bootstrap.cc:492] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:25.336414 5213 raft_consensus.cc:2272] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:25.337296 4644 tablet_service.cc:1511] Processing CreateTablet for tablet 0827e2c7eff041369a94c31abdf9b093 (DEFAULT_TABLE table=default.table_127c3d62fd1c4394bcd55bc6fc18bc5d [id=45cbddfbaced49a989c1fb61a644a713]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.337734 4644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0827e2c7eff041369a94c31abdf9b093. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:25.337756 4785 tablet_service.cc:1511] Processing CreateTablet for tablet 0827e2c7eff041369a94c31abdf9b093 (DEFAULT_TABLE table=default.table_127c3d62fd1c4394bcd55bc6fc18bc5d [id=45cbddfbaced49a989c1fb61a644a713]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.338130 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0827e2c7eff041369a94c31abdf9b093. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:25.338567 5216 ts_tablet_manager.cc:1929] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:25.338711 5216 log.cc:1199] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/88f6f526af054da1973d202e14ce22ae
I20260925 20:00:25.339186 5216 ts_tablet_manager.cc:1950] T 88f6f526af054da1973d202e14ce22ae P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:25.340703 4506 tablet_service.cc:1511] Processing CreateTablet for tablet 0827e2c7eff041369a94c31abdf9b093 (DEFAULT_TABLE table=default.table_127c3d62fd1c4394bcd55bc6fc18bc5d [id=45cbddfbaced49a989c1fb61a644a713]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.341110 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0827e2c7eff041369a94c31abdf9b093. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:25.343889 5216 tablet_replica.cc:333] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:25.344090 5216 raft_consensus.cc:2243] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:25.344448 5216 raft_consensus.cc:2272] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:25.345031 5216 ts_tablet_manager.cc:1916] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:25.345950 4314 catalog_manager.cc:5063] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): tablet 88f6f526af054da1973d202e14ce22ae (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]) successfully deleted
I20260925 20:00:25.348665 5213 ts_tablet_manager.cc:1916] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:25.350987 4326 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e: default.table_703616ef44ba43f9afb9c4df73eb372c [id=25f6e1ade43c4519bed519037ee8f761] alter complete (version 1)
I20260925 20:00:25.357463 4326 catalog_manager.cc:5938] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395) does not have the latest schema for tablet 72c3c64108f348a5ab893e6954d72d77 (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]). Expected version 1 got 0
I20260925 20:00:25.361481 5218 tablet_bootstrap.cc:492] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:25.363699 5218 tablet_bootstrap.cc:654] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:25.363903 5216 ts_tablet_manager.cc:1929] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:25.364050 5216 log.cc:1199] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/635a8a28ca8444dd8162e29a405810d6
I20260925 20:00:25.364552 5216 ts_tablet_manager.cc:1950] T 635a8a28ca8444dd8162e29a405810d6 P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:25.365818 4314 catalog_manager.cc:5063] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): tablet 635a8a28ca8444dd8162e29a405810d6 (table default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf]) successfully deleted
I20260925 20:00:25.367844 5213 ts_tablet_manager.cc:1929] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:25.370404 5228 tablet_bootstrap.cc:654] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:25.373138 5213 log.cc:1199] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/635a8a28ca8444dd8162e29a405810d6
I20260925 20:00:25.374035 5213 ts_tablet_manager.cc:1950] T 635a8a28ca8444dd8162e29a405810d6 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:25.374900 5218 tablet_bootstrap.cc:492] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:25.375058 5218 ts_tablet_manager.cc:1403] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.003s
I20260925 20:00:25.375824 5218 raft_consensus.cc:359] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.376039 5218 raft_consensus.cc:385] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:25.376142 5218 raft_consensus.cc:740] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:25.376657 5223 ts_tablet_manager.cc:1929] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:25.378232 5223 log.cc:1199] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/88f6f526af054da1973d202e14ce22ae
I20260925 20:00:25.378662 5223 ts_tablet_manager.cc:1950] T 88f6f526af054da1973d202e14ce22ae P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:25.379704 4316 catalog_manager.cc:5063] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): tablet 88f6f526af054da1973d202e14ce22ae (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]) successfully deleted
I20260925 20:00:25.379814 5218 consensus_queue.cc:260] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.379997 4316 catalog_manager.cc:5063] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395): tablet 635a8a28ca8444dd8162e29a405810d6 (table default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf]) successfully deleted
I20260925 20:00:25.380622 4326 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 7f43a6a14ef547f2aeb22ce2b334383e: default.table_098b200909f842c88a38f11192f4dc25 [id=86f1c8a16cc14ef9b72c95060313b3ad] alter complete (version 1)
I20260925 20:00:25.380960 5213 tablet_replica.cc:333] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
W20260925 20:00:25.381248 4326 catalog_manager.cc:6038] Failed to send AlterTable RPC for tablet 72c3c64108f348a5ab893e6954d72d77 (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 72c3c64108f348a5ab893e6954d72d77 consensus state has no leader
I20260925 20:00:25.381592 5213 raft_consensus.cc:2243] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:25.382544 5213 raft_consensus.cc:2272] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:25.385596 5213 ts_tablet_manager.cc:1916] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:25.387831 4326 catalog_manager.cc:5938] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647) does not have the latest schema for tablet 72c3c64108f348a5ab893e6954d72d77 (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]). Expected version 1 got 0
I20260925 20:00:25.388054 5218 ts_tablet_manager.cc:1434] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.013s user 0.001s sys 0.002s
I20260925 20:00:25.388993 5228 tablet_bootstrap.cc:492] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:25.395009 5192 consensus_queue.cc:1048] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:25.395361 5228 ts_tablet_manager.cc:1403] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.060s user 0.002s sys 0.003s
I20260925 20:00:25.395495 5192 consensus_queue.cc:1048] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:25.396358 5228 raft_consensus.cc:359] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.396883 5228 raft_consensus.cc:385] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:25.397068 5228 raft_consensus.cc:740] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:25.394842 5223 tablet_replica.cc:333] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:25.397576 5223 raft_consensus.cc:2243] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:25.397827 5223 raft_consensus.cc:2272] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:25.398556 5223 ts_tablet_manager.cc:1916] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:25.407433 5226 tablet.cc:1751] T ee843d55757347ef9942dffd917c0ff8 P 145cdcf7953e4c0d9686b1c6a86f9482: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 20:00:25.408515 5228 consensus_queue.cc:260] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.410802 5228 ts_tablet_manager.cc:1434] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.015s user 0.003s sys 0.000s
I20260925 20:00:25.411515 5228 tablet_bootstrap.cc:492] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:25.414073 5228 tablet_bootstrap.cc:654] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:25.415613 5223 ts_tablet_manager.cc:1929] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:25.415907 5223 log.cc:1199] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/635a8a28ca8444dd8162e29a405810d6
I20260925 20:00:25.416589 5223 ts_tablet_manager.cc:1950] T 635a8a28ca8444dd8162e29a405810d6 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:25.418689 4316 catalog_manager.cc:5063] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): tablet 635a8a28ca8444dd8162e29a405810d6 (table default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf]) successfully deleted
I20260925 20:00:25.423545 5228 tablet_bootstrap.cc:492] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:25.423841 5228 ts_tablet_manager.cc:1403] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.001s
I20260925 20:00:25.424630 4188 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=44287e882a4040dcb79d824c08808dd4]
I20260925 20:00:25.425060 4188 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48108 (request call id 49) took 4116 ms. Trace:
I20260925 20:00:25.425278 5228 raft_consensus.cc:359] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.425212 4188 rpcz_store.cc:276] 0925 20:00:21.308986 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:21.434775 (+125789us) service_pool.cc:224] Handling call
0925 20:00:22.824709 (+1389934us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 20:00:22.825276 (+ 567us) catalog_manager.cc:2186] Created new table and tablet info
0925 20:00:24.884971 (+2059695us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 20:00:24.886018 (+ 1047us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:24.886024 (+ 6us) write_op.cc:454] Released schema lock
0925 20:00:24.886351 (+ 327us) catalog_manager.cc:2220] Rolling back HMS table creation
0925 20:00:25.425009 (+538658us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 20:00:24.885880 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica 169b6de20560440b99cc08af549ce477 is not leader of this config: current role FOLLOWER)
0925 20:00:24.885902 (+ 22us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:24.885913 (+ 11us) write_op.cc:454] Released schema lock
0925 20:00:24.885921 (+ 8us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1938104,"Hive Metastore.run_cpu_time_us":578,"Hive Metastore.run_wall_time_us":658314,"child_traces":[["op",{}]]}
I20260925 20:00:25.425900 5228 raft_consensus.cc:385] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:25.426120 5228 raft_consensus.cc:740] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:25.427186 5228 consensus_queue.cc:260] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.428467 5217 tablet_bootstrap.cc:492] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:25.428792 5228 ts_tablet_manager.cc:1434] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260925 20:00:25.429275 5217 ts_tablet_manager.cc:1403] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.123s user 0.006s sys 0.013s
W20260925 20:00:25.432626 4315 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 72c3c64108f348a5ab893e6954d72d77 (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 72c3c64108f348a5ab893e6954d72d77 consensus state has no leader
I20260925 20:00:25.440068 4326 catalog_manager.cc:5938] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395) does not have the latest schema for tablet 72c3c64108f348a5ab893e6954d72d77 (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]). Expected version 1 got 0
I20260925 20:00:25.443110 5217 raft_consensus.cc:359] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.443588 5217 raft_consensus.cc:385] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:25.443750 5217 raft_consensus.cc:740] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:25.444170 5217 consensus_queue.cc:260] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.445093 5217 ts_tablet_manager.cc:1434] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260925 20:00:25.445554 5217 tablet_bootstrap.cc:492] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:25.445955 4326 catalog_manager.cc:5938] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741) does not have the latest schema for tablet 72c3c64108f348a5ab893e6954d72d77 (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]). Expected version 1 got 0
I20260925 20:00:25.454535 5217 tablet_bootstrap.cc:654] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:25.463933 4326 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52966:
table { table_name: "default.Table_e72a162b7a554904a9ca0cceb5e6b273" } new_table_name: "default.Table_1073b55979e5429c83265f7d64835607" modify_external_catalogs: true
I20260925 20:00:25.461035 5213 ts_tablet_manager.cc:1929] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:25.465582 5213 log.cc:1199] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/988fd177aaf3418d89c3eb5756f989ba
I20260925 20:00:25.466228 5213 ts_tablet_manager.cc:1950] T 988fd177aaf3418d89c3eb5756f989ba P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:25.468909 4316 catalog_manager.cc:5063] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395): tablet 988fd177aaf3418d89c3eb5756f989ba (table default.table_4ec42ee954664a658b3aad36206c53aa [id=c47fb6987d0343418ed55e663b34ab74]) successfully deleted
I20260925 20:00:25.485801 5217 tablet_bootstrap.cc:492] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:25.486104 5217 ts_tablet_manager.cc:1403] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.041s user 0.004s sys 0.001s
I20260925 20:00:25.487216 5217 raft_consensus.cc:359] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.487581 5217 raft_consensus.cc:385] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:25.487743 5217 raft_consensus.cc:740] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:25.488396 5217 consensus_queue.cc:260] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.494395 5217 ts_tablet_manager.cc:1434] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260925 20:00:25.495976 5187 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:25.496182 5187 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:25.496974 5187 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:25.497709 5132 raft_consensus.cc:493] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:25.497784 4804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c3c64108f348a5ab893e6954d72d77" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" is_pre_election: true
I20260925 20:00:25.497889 5132 raft_consensus.cc:515] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.498075 4804 raft_consensus.cc:2468] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:25.498481 5132 leader_election.cc:290] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:25.498589 4666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c3c64108f348a5ab893e6954d72d77" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:25.498889 4666 raft_consensus.cc:2468] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:25.499164 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f74f459c542a88e8bef5f1c51357d" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:25.499398 4526 raft_consensus.cc:2468] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:25.499816 4455 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:25.500227 5187 raft_consensus.cc:2804] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:25.500392 5187 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:25.500492 5187 raft_consensus.cc:3060] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:25.500824 4741 leader_election.cc:304] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:25.502041 4666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f74f459c542a88e8bef5f1c51357d" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:25.502285 4666 raft_consensus.cc:2468] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:25.503435 5132 raft_consensus.cc:2804] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:25.503800 5132 raft_consensus.cc:493] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:25.504509 5132 raft_consensus.cc:3060] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:25.504192 5187 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:25.507686 5187 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Requested vote from peers dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:25.508320 4804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c3c64108f348a5ab893e6954d72d77" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
I20260925 20:00:25.508344 4666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72c3c64108f348a5ab893e6954d72d77" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:25.508548 4804 raft_consensus.cc:3060] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:25.508627 4666 raft_consensus.cc:3060] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:25.511907 4804 raft_consensus.cc:2468] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
I20260925 20:00:25.512538 4457 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:25.515412 4666 raft_consensus.cc:2468] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
I20260925 20:00:25.512156 5132 raft_consensus.cc:515] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.519088 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f74f459c542a88e8bef5f1c51357d" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:25.519326 4526 raft_consensus.cc:3060] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:25.522468 4526 raft_consensus.cc:2468] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:25.523221 4741 leader_election.cc:304] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:25.523664 5143 raft_consensus.cc:2804] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:25.523818 5143 raft_consensus.cc:697] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Becoming Leader. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Running, Role: LEADER
I20260925 20:00:25.524158 5143 consensus_queue.cc:237] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.527772 4666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b59f74f459c542a88e8bef5f1c51357d" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:25.528475 4666 raft_consensus.cc:3060] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:25.530937 5187 raft_consensus.cc:2804] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:25.530614 5132 leader_election.cc:290] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Requested vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:25.535310 4666 raft_consensus.cc:2468] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:25.535454 4328 catalog_manager.cc:2213] created HMS catalog entry for table [id=a385e0b7db6a49e8b1baf34a5fdd3473]
I20260925 20:00:25.563333 4506 tablet_service.cc:1511] Processing CreateTablet for tablet 653a4121e8934471b99a9853b73617ab (DEFAULT_TABLE table=default.table_5b8386185e8e4fcd99110bd5137a578c [id=a385e0b7db6a49e8b1baf34a5fdd3473]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.563648 4644 tablet_service.cc:1511] Processing CreateTablet for tablet 653a4121e8934471b99a9853b73617ab (DEFAULT_TABLE table=default.table_5b8386185e8e4fcd99110bd5137a578c [id=a385e0b7db6a49e8b1baf34a5fdd3473]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.563876 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 653a4121e8934471b99a9853b73617ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:25.564092 4644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 653a4121e8934471b99a9853b73617ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:25.567270 4785 tablet_service.cc:1511] Processing CreateTablet for tablet 653a4121e8934471b99a9853b73617ab (DEFAULT_TABLE table=default.table_5b8386185e8e4fcd99110bd5137a578c [id=a385e0b7db6a49e8b1baf34a5fdd3473]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.567730 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 653a4121e8934471b99a9853b73617ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:25.577451 4328 catalog_manager.cc:5838] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e reported cstate change: term changed from 0 to 1, leader changed from <none> to dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195). New cstate: current_term: 1 leader_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: HEALTHY } } }
I20260925 20:00:25.579895 5218 tablet_bootstrap.cc:492] T 653a4121e8934471b99a9853b73617ab P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:25.582047 5218 tablet_bootstrap.cc:654] T 653a4121e8934471b99a9853b73617ab P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:25.585685 5217 tablet_bootstrap.cc:492] T 653a4121e8934471b99a9853b73617ab P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:25.588006 5217 tablet_bootstrap.cc:654] T 653a4121e8934471b99a9853b73617ab P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:25.591812 5218 tablet_bootstrap.cc:492] T 653a4121e8934471b99a9853b73617ab P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:25.592617 5218 ts_tablet_manager.cc:1403] T 653a4121e8934471b99a9853b73617ab P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260925 20:00:25.593539 5218 raft_consensus.cc:359] T 653a4121e8934471b99a9853b73617ab P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.593896 5218 raft_consensus.cc:385] T 653a4121e8934471b99a9853b73617ab P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:25.594048 5218 raft_consensus.cc:740] T 653a4121e8934471b99a9853b73617ab P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:25.594425 5218 consensus_queue.cc:260] T 653a4121e8934471b99a9853b73617ab P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.598503 5228 tablet_bootstrap.cc:492] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:25.600829 5228 tablet_bootstrap.cc:654] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:25.601434 5218 ts_tablet_manager.cc:1434] T 653a4121e8934471b99a9853b73617ab P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260925 20:00:25.613524 5228 tablet_bootstrap.cc:492] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:25.613775 5228 ts_tablet_manager.cc:1403] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.004s
I20260925 20:00:25.614842 5228 raft_consensus.cc:359] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.615108 5228 raft_consensus.cc:385] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:25.615200 5228 raft_consensus.cc:740] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:25.615563 5228 consensus_queue.cc:260] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.616158 4328 master_service.cc:960] ReplaceTablet: received request to replace tablet dff315b3b8e64ed0838b6241eeb442d3 from {username='slave'} at 127.0.0.1:52966
I20260925 20:00:25.617614 4328 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52966:
table { table_name: "default.Table_c78f9cc1d7e74b51b29858c7d3248fb8" } new_table_name: "default.Table_1f01c68bbe404907a4d5f4d12fb9ee26" modify_external_catalogs: true
I20260925 20:00:25.621642 5217 tablet_bootstrap.cc:492] T 653a4121e8934471b99a9853b73617ab P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:25.621910 5217 ts_tablet_manager.cc:1403] T 653a4121e8934471b99a9853b73617ab P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.006s
I20260925 20:00:25.622736 5217 raft_consensus.cc:359] T 653a4121e8934471b99a9853b73617ab P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.623039 5217 raft_consensus.cc:385] T 653a4121e8934471b99a9853b73617ab P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:25.623142 5217 raft_consensus.cc:740] T 653a4121e8934471b99a9853b73617ab P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:25.623430 5217 consensus_queue.cc:260] T 653a4121e8934471b99a9853b73617ab P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.624097 5217 ts_tablet_manager.cc:1434] T 653a4121e8934471b99a9853b73617ab P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260925 20:00:25.625450 5228 ts_tablet_manager.cc:1434] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.012s user 0.002s sys 0.002s
I20260925 20:00:25.640377 5187 raft_consensus.cc:697] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 LEADER]: Becoming Leader. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Running, Role: LEADER
I20260925 20:00:25.640853 5187 consensus_queue.cc:237] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:25.655871 4183 catalog_manager.cc:2770] deleted HMS catalog entry for table default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330]
I20260925 20:00:25.656685 4183 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:48108 (request call id 64) took 3632 ms. Trace:
I20260925 20:00:25.656821 4183 rpcz_store.cc:276] 0925 20:00:22.024078 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:23.547565 (+1523487us) service_pool.cc:224] Handling call
0925 20:00:24.925946 (+1378381us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 20:00:25.656611 (+730665us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":498340,"Hive Metastore.run_cpu_time_us":238,"Hive Metastore.run_wall_time_us":231143}
I20260925 20:00:25.674032 4327 catalog_manager.cc:2213] created HMS catalog entry for table [id=439b3d65790141d1abfbca4788836f80]
I20260925 20:00:25.688079 4327 catalog_manager.cc:5838] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 reported cstate change: term changed from 0 to 1, leader changed from <none> to 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193). New cstate: current_term: 1 leader_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260925 20:00:25.693295 4327 catalog_manager.cc:5938] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741) does not have the latest schema for tablet 72c3c64108f348a5ab893e6954d72d77 (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]). Expected version 1 got 0
I20260925 20:00:25.703384 5187 raft_consensus.cc:493] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:25.703625 5187 raft_consensus.cc:515] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.704384 5187 leader_election.cc:290] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:25.705143 4666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "653a4121e8934471b99a9853b73617ab" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:25.705413 4666 raft_consensus.cc:2468] T 653a4121e8934471b99a9853b73617ab P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:25.706631 4804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "653a4121e8934471b99a9853b73617ab" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" is_pre_election: true
I20260925 20:00:25.707011 4804 raft_consensus.cc:2468] T 653a4121e8934471b99a9853b73617ab P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:25.707764 4455 leader_election.cc:304] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:25.709219 5187 raft_consensus.cc:2804] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:25.709326 5187 raft_consensus.cc:493] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:25.709427 5187 raft_consensus.cc:3060] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:25.712357 5187 raft_consensus.cc:515] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.713276 4329 catalog_manager.cc:2213] created HMS catalog entry for table [id=4b80e3afa6954e2ea8b6d1b5ec79a9d1]
I20260925 20:00:25.719133 4327 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52966:
table { table_name: "default.Table_098b200909f842c88a38f11192f4dc25" } new_table_name: "default.Table_12cc527a0517451f9a6f226194c4ca68" modify_external_catalogs: true
F20260925 20:00:25.724633 4385 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 21, previous 21; dumping first 2 out of 6 received events: NotificationEvent(eventId=21, eventTime=1790366425, eventType=DROP_TABLE, dbName=default, tableName=table_30a6c0d501b04bfc882cec2f43c06f83, message={"server":"","servicePrincipal":"","db":"default","table":"table_30a6c0d501b04bfc882cec2f43c06f83","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_30a6c0d501b04bfc882cec2f43c06f83\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790366424},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/warehouse/table_30a6c0d501b04bfc882cec2f43c06f83\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_30a6c0d501b04bfc882cec2f43c06f83\",\"kudu.table_id\":\"44287e882a4040dcb79d824c08808dd4\",\"transient_lastDdlTime\":\"1790366424\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"7efb83138ca94380bf2d8f0f2f91c381\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790366425}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=21, eventTime=1790366425, eventType=CREATE_TABLE, dbName=default, tableName=table_127c3d62fd1c4394bcd55bc6fc18bc5d, message={"server":"","servicePrincipal":"","db":"default","table":"table_127c3d62fd1c4394bcd55bc6fc18bc5d","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_127c3d62fd1c4394bcd55bc6fc18bc5d\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790366425},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/warehouse/table_127c3d62fd1c4394bcd55bc6fc18bc5d\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_127c3d62fd1c4394bcd55bc6fc18bc5d\",\"kudu.table_id\":\"45cbddfbaced49a989c1fb61a644a713\",\"transient_lastDdlTime\":\"1790366425\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"7efb83138ca94380bf2d8f0f2f91c381\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790366425,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f487f392d5d google::LogMessage::Fail() at ??:0
@ 0x7f487f396b23 google::LogMessage::SendToLog() at ??:0
@ 0x7f487f39275c google::LogMessage::Flush() at ??:0
@ 0x7f487f3929a1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f48960912f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260925 20:00:25.744037 4506 tablet_service.cc:1511] Processing CreateTablet for tablet b65400989b944ca988d6f26838277336 (DEFAULT_TABLE table=default.table_667eace0e712438bbd1211f7e3e665f8 [id=439b3d65790141d1abfbca4788836f80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.744577 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65400989b944ca988d6f26838277336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:25.746812 4785 tablet_service.cc:1511] Processing CreateTablet for tablet b65400989b944ca988d6f26838277336 (DEFAULT_TABLE table=default.table_667eace0e712438bbd1211f7e3e665f8 [id=439b3d65790141d1abfbca4788836f80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.747298 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65400989b944ca988d6f26838277336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:25.750911 5228 tablet_bootstrap.cc:492] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:25.751163 5244 raft_consensus.cc:493] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:25.751344 5244 raft_consensus.cc:515] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.752079 4644 tablet_service.cc:1511] Processing CreateTablet for tablet b65400989b944ca988d6f26838277336 (DEFAULT_TABLE table=default.table_667eace0e712438bbd1211f7e3e665f8 [id=439b3d65790141d1abfbca4788836f80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.752205 5244 leader_election.cc:290] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:25.752559 4644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65400989b944ca988d6f26838277336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:25.752777 5228 tablet_bootstrap.cc:654] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:25.754315 4804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0827e2c7eff041369a94c31abdf9b093" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" is_pre_election: true
I20260925 20:00:25.754544 4804 raft_consensus.cc:2468] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:25.755076 4457 leader_election.cc:304] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:25.755419 5244 raft_consensus.cc:2804] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:25.755525 5244 raft_consensus.cc:493] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:25.755604 5244 raft_consensus.cc:3060] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:25.758594 5244 raft_consensus.cc:515] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.758666 4666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0827e2c7eff041369a94c31abdf9b093" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:25.758924 4666 raft_consensus.cc:2468] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:25.759670 5244 leader_election.cc:290] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Requested vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:25.759927 4666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0827e2c7eff041369a94c31abdf9b093" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:25.760106 4666 raft_consensus.cc:3060] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:25.760362 4804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0827e2c7eff041369a94c31abdf9b093" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
I20260925 20:00:25.761006 4804 raft_consensus.cc:3060] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:25.763810 4666 raft_consensus.cc:2468] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
I20260925 20:00:25.764252 4455 leader_election.cc:304] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:25.764611 5244 raft_consensus.cc:2804] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:25.765134 5244 raft_consensus.cc:697] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 LEADER]: Becoming Leader. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Running, Role: LEADER
I20260925 20:00:25.765470 5244 consensus_queue.cc:237] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
@ 0x7f489608fba1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260925 20:00:25.778081 4805 raft_consensus.cc:1275] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Refusing update from remote peer 145cdcf7953e4c0d9686b1c6a86f9482: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 20:00:25.778890 5248 consensus_queue.cc:1048] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:25.779886 4325 catalog_manager.cc:2770] deleted HMS catalog entry for table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]
I20260925 20:00:25.783488 4804 raft_consensus.cc:2468] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
I20260925 20:00:25.785317 4665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "653a4121e8934471b99a9853b73617ab" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:25.785563 4665 raft_consensus.cc:3060] T 653a4121e8934471b99a9853b73617ab P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
@ 0x7f489609781c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260925 20:00:25.788781 5187 leader_election.cc:290] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Requested vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:25.790004 4665 raft_consensus.cc:2468] T 653a4121e8934471b99a9853b73617ab P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
I20260925 20:00:25.790269 4803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "653a4121e8934471b99a9853b73617ab" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
I20260925 20:00:25.790498 4803 raft_consensus.cc:3060] T 653a4121e8934471b99a9853b73617ab P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:25.790525 4455 leader_election.cc:304] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:25.793663 4803 raft_consensus.cc:2468] T 653a4121e8934471b99a9853b73617ab P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
I20260925 20:00:25.796376 5218 tablet_bootstrap.cc:492] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:25.797885 4329 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52966:
table { table_name: "default.Table_cf7560049cf84f22b27dd94a6b4dbcef" } modify_external_catalogs: true
I20260925 20:00:25.798130 4329 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52966:
table { table_name: "default.Table_cf7560049cf84f22b27dd94a6b4dbcef" } modify_external_catalogs: true
I20260925 20:00:25.798609 5218 tablet_bootstrap.cc:654] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:25.802312 4784 tablet_service.cc:1511] Processing CreateTablet for tablet 5982cfa34e4e40d3894cbf46aab33715 (DEFAULT_TABLE table=default.table_2ebd7ecb76b44d13a6847408f7fd7d17 [id=4b80e3afa6954e2ea8b6d1b5ec79a9d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.802966 4784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5982cfa34e4e40d3894cbf46aab33715. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:25.810436 4666 raft_consensus.cc:1275] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Refusing update from remote peer 145cdcf7953e4c0d9686b1c6a86f9482: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 20:00:25.811367 5248 raft_consensus.cc:2804] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:25.811576 5248 raft_consensus.cc:697] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 LEADER]: Becoming Leader. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Running, Role: LEADER
I20260925 20:00:25.811981 5248 consensus_queue.cc:237] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.814319 5187 consensus_queue.cc:1048] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:25.820978 5226 tablet.cc:1751] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
@ 0x7f4896097699 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xaaef9d std::function<>::operator()() at ??:?
I20260925 20:00:25.838665 4506 tablet_service.cc:1511] Processing CreateTablet for tablet 5982cfa34e4e40d3894cbf46aab33715 (DEFAULT_TABLE table=default.table_2ebd7ecb76b44d13a6847408f7fd7d17 [id=4b80e3afa6954e2ea8b6d1b5ec79a9d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.839244 5217 tablet_bootstrap.cc:492] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:25.839578 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5982cfa34e4e40d3894cbf46aab33715. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f48820e2079 kudu::Thread::SuperviseThread() at ??:0
I20260925 20:00:25.841125 5217 tablet_bootstrap.cc:654] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:25.842849 4644 tablet_service.cc:1511] Processing CreateTablet for tablet 5982cfa34e4e40d3894cbf46aab33715 (DEFAULT_TABLE table=default.table_2ebd7ecb76b44d13a6847408f7fd7d17 [id=4b80e3afa6954e2ea8b6d1b5ec79a9d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:25.843353 4644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5982cfa34e4e40d3894cbf46aab33715. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:25.848932 5259 tablet.cc:1751] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
@ 0x7f48857b86db start_thread at ??:0
@ 0x7f487e23f61f clone at ??:0
I20260925 20:00:25.863692 5217 tablet_bootstrap.cc:492] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:25.863929 5217 ts_tablet_manager.cc:1403] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.025s user 0.006s sys 0.005s
I20260925 20:00:25.864959 5217 raft_consensus.cc:359] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.865211 5217 raft_consensus.cc:385] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:25.865298 5217 raft_consensus.cc:740] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:25.865705 5217 consensus_queue.cc:260] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.876775 5224 tablet.cc:1751] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 20:00:25.878716 4184 catalog_manager.cc:2770] deleted HMS catalog entry for table default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf]
I20260925 20:00:25.879725 4184 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:48108 (request call id 65) took 3754 ms. Trace:
I20260925 20:00:25.879873 4184 rpcz_store.cc:276] 0925 20:00:22.125054 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:23.548331 (+1423277us) service_pool.cc:224] Handling call
0925 20:00:24.926107 (+1377776us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 20:00:25.879649 (+953542us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":729438,"Hive Metastore.run_cpu_time_us":279,"Hive Metastore.run_wall_time_us":222822}
I20260925 20:00:25.899632 5217 ts_tablet_manager.cc:1434] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.036s user 0.003s sys 0.000s
I20260925 20:00:25.904238 5217 tablet_bootstrap.cc:492] T 5982cfa34e4e40d3894cbf46aab33715 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:25.906294 5217 tablet_bootstrap.cc:654] T 5982cfa34e4e40d3894cbf46aab33715 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:25.940853 5217 tablet_bootstrap.cc:492] T 5982cfa34e4e40d3894cbf46aab33715 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:25.941061 5217 ts_tablet_manager.cc:1403] T 5982cfa34e4e40d3894cbf46aab33715 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.037s user 0.000s sys 0.003s
I20260925 20:00:25.941210 5228 tablet_bootstrap.cc:492] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:25.941365 5228 ts_tablet_manager.cc:1403] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.191s user 0.010s sys 0.029s
I20260925 20:00:25.941967 5217 raft_consensus.cc:359] T 5982cfa34e4e40d3894cbf46aab33715 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.942188 5217 raft_consensus.cc:385] T 5982cfa34e4e40d3894cbf46aab33715 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:25.942090 5228 raft_consensus.cc:359] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.942269 5217 raft_consensus.cc:740] T 5982cfa34e4e40d3894cbf46aab33715 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:25.942283 5228 raft_consensus.cc:385] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:25.942368 5228 raft_consensus.cc:740] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:25.942653 5228 consensus_queue.cc:260] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.943356 5228 ts_tablet_manager.cc:1434] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260925 20:00:25.947501 5217 consensus_queue.cc:260] T 5982cfa34e4e40d3894cbf46aab33715 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.953464 5217 ts_tablet_manager.cc:1434] T 5982cfa34e4e40d3894cbf46aab33715 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260925 20:00:25.956112 5228 tablet_bootstrap.cc:492] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:25.958344 5228 tablet_bootstrap.cc:654] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:25.970332 5218 tablet_bootstrap.cc:492] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:25.970551 5218 ts_tablet_manager.cc:1403] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.174s user 0.015s sys 0.011s
I20260925 20:00:25.971635 5218 raft_consensus.cc:359] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.971899 5218 raft_consensus.cc:385] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:25.972004 5218 raft_consensus.cc:740] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:25.972425 5218 consensus_queue.cc:260] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:25.981060 5218 ts_tablet_manager.cc:1434] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260925 20:00:25.989229 5218 tablet_bootstrap.cc:492] T 5982cfa34e4e40d3894cbf46aab33715 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:25.991750 5218 tablet_bootstrap.cc:654] T 5982cfa34e4e40d3894cbf46aab33715 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:26.001588 5228 tablet_bootstrap.cc:492] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:26.001781 5228 ts_tablet_manager.cc:1403] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.046s user 0.018s sys 0.025s
I20260925 20:00:26.002602 5228 raft_consensus.cc:359] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:26.002799 5228 raft_consensus.cc:385] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:26.002866 5228 raft_consensus.cc:740] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:26.003171 5228 consensus_queue.cc:260] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:26.005882 5228 ts_tablet_manager.cc:1434] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260925 20:00:26.009498 5218 tablet_bootstrap.cc:492] T 5982cfa34e4e40d3894cbf46aab33715 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:26.009685 5218 ts_tablet_manager.cc:1403] T 5982cfa34e4e40d3894cbf46aab33715 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.000s
I20260925 20:00:26.010524 5218 raft_consensus.cc:359] T 5982cfa34e4e40d3894cbf46aab33715 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:26.010710 5218 raft_consensus.cc:385] T 5982cfa34e4e40d3894cbf46aab33715 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:26.010778 5218 raft_consensus.cc:740] T 5982cfa34e4e40d3894cbf46aab33715 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:26.011034 5218 consensus_queue.cc:260] T 5982cfa34e4e40d3894cbf46aab33715 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:26.024925 5218 ts_tablet_manager.cc:1434] T 5982cfa34e4e40d3894cbf46aab33715 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260925 20:00:26.076534 4180 catalog_manager.cc:2770] deleted HMS catalog entry for table default.table_7722a249d10f4616ba5b5bb2fc149c18 [id=a8f568d7faab469ba7ea8a2a38bfd638]
I20260925 20:00:26.077069 4180 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:48108 (request call id 66) took 3902 ms. Trace:
I20260925 20:00:26.077183 4180 rpcz_store.cc:276] 0925 20:00:22.174484 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:23.963104 (+1788620us) service_pool.cc:224] Handling call
0925 20:00:24.926742 (+963638us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 20:00:26.077032 (+1150290us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":951683,"Hive Metastore.run_cpu_time_us":208,"Hive Metastore.run_wall_time_us":197636}
I20260925 20:00:26.116134 4187 catalog_manager.cc:2213] created HMS catalog entry for table [id=a0c689dec3f74cf080ff016d9c7a2558]
W20260925 20:00:26.117179 4187 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica 169b6de20560440b99cc08af549ce477 is not leader of this config: current role FOLLOWER
I20260925 20:00:26.125084 5248 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:26.125324 5248 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:26.126106 5248 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:26.126957 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65400989b944ca988d6f26838277336" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" is_pre_election: true
I20260925 20:00:26.127226 4805 raft_consensus.cc:2468] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:26.127393 4665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65400989b944ca988d6f26838277336" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:26.127614 4665 raft_consensus.cc:2468] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:26.128087 4457 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:26.128404 5248 raft_consensus.cc:2804] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:26.128515 5248 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:26.128608 5248 raft_consensus.cc:3060] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:26.131735 5248 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:26.132970 4665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65400989b944ca988d6f26838277336" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:26.133230 4665 raft_consensus.cc:3060] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:26.136534 4665 raft_consensus.cc:2468] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
I20260925 20:00:26.137291 4455 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:26.137770 5187 raft_consensus.cc:2804] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:26.137940 5187 raft_consensus.cc:697] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 LEADER]: Becoming Leader. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Running, Role: LEADER
I20260925 20:00:26.138232 5187 consensus_queue.cc:237] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:26.144634 5248 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Requested vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:26.145218 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65400989b944ca988d6f26838277336" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
I20260925 20:00:26.145437 4805 raft_consensus.cc:3060] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:26.148639 4805 raft_consensus.cc:2468] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
I20260925 20:00:26.160749 4189 catalog_manager.cc:2213] created HMS catalog entry for table [id=f99d2f0c40a8489299b9036d342781ef]
I20260925 20:00:26.161188 3871 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu pid 4936
W20260925 20:00:26.161660 4189 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica 169b6de20560440b99cc08af549ce477 is not leader of this config: current role FOLLOWER
I20260925 20:00:26.196599 5092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 20:00:26.229873 4187 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=a0c689dec3f74cf080ff016d9c7a2558]
I20260925 20:00:26.230329 4187 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48108 (request call id 63) took 4288 ms. Trace:
I20260925 20:00:26.230432 4187 rpcz_store.cc:276] 0925 20:00:21.941767 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:22.950297 (+1008530us) service_pool.cc:224] Handling call
0925 20:00:24.955139 (+2004842us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 20:00:24.955683 (+ 544us) catalog_manager.cc:2186] Created new table and tablet info
0925 20:00:26.116097 (+1160414us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 20:00:26.116992 (+ 895us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:26.116997 (+ 5us) write_op.cc:454] Released schema lock
0925 20:00:26.117311 (+ 314us) catalog_manager.cc:2220] Rolling back HMS table creation
0925 20:00:26.230251 (+112940us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 20:00:26.116911 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica 169b6de20560440b99cc08af549ce477 is not leader of this config: current role FOLLOWER)
0925 20:00:26.116925 (+ 14us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:26.116931 (+ 6us) write_op.cc:454] Released schema lock
0925 20:00:26.116936 (+ 5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1163504,"Hive Metastore.run_cpu_time_us":604,"Hive Metastore.run_wall_time_us":108629,"child_traces":[["op",{}]]}
W20260925 20:00:26.232012 4862 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.199.252:37957)
W20260925 20:00:26.287859 4740 connection.cc:573] server connection from 127.0.0.1:54530 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:26.287886 4601 connection.cc:573] server connection from 127.0.0.1:57394 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:26.287837 4457 connection.cc:573] server connection from 127.0.0.1:54960 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:26.288764 4863 connection.cc:573] client connection to 127.3.199.252:37957 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:26.289292 4863 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.3.199.252:37957)
W20260925 20:00:26.290102 4863 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.3.199.252:37957)
W20260925 20:00:26.291216 4863 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.199.252:37957)
W20260925 20:00:26.305284 5281 negotiation.cc:336] Failed RPC negotiation. Trace:
0925 20:00:26.294307 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.199.252:37957 (local address 127.3.199.195:51227)
0925 20:00:26.295519 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0925 20:00:26.295540 (+ 21us) client_negotiation.cc:175] Beginning negotiation
0925 20:00:26.295685 (+ 145us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0925 20:00:26.304250 (+ 8565us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1044,"thread_start_us":897,"threads_started":1}
W20260925 20:00:26.306602 5283 negotiation.cc:336] Failed RPC negotiation. Trace:
0925 20:00:26.301048 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.199.252:37957 (local address 127.0.0.1:54132)
0925 20:00:26.306042 (+ 4994us) negotiation.cc:107] Waiting for socket to connect
0925 20:00:26.306229 (+ 187us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":4792,"threads_started":1}
W20260925 20:00:26.308317 5282 negotiation.cc:336] Failed RPC negotiation. Trace:
0925 20:00:26.296612 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.3.199.252:37957 (local address 127.3.199.194:37971)
0925 20:00:26.300463 (+ 3851us) negotiation.cc:107] Waiting for socket to connect
0925 20:00:26.300487 (+ 24us) client_negotiation.cc:175] Beginning negotiation
0925 20:00:26.300639 (+ 152us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0925 20:00:26.306801 (+ 6162us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":3712,"thread_start_us":3903,"threads_started":1}
I20260925 20:00:26.309001 5280 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7f43a6a14ef547f2aeb22ce2b334383e)
I20260925 20:00:26.310452 5280 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:26.311434 5280 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:26.315265 4189 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=f99d2f0c40a8489299b9036d342781ef]
I20260925 20:00:26.315649 4189 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48108 (request call id 62) took 4520 ms. Trace:
I20260925 20:00:26.315788 4189 rpcz_store.cc:276] 0925 20:00:21.795070 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 20:00:22.947584 (+1152514us) service_pool.cc:224] Handling call
0925 20:00:24.968599 (+2021015us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 20:00:24.969149 (+ 550us) catalog_manager.cc:2186] Created new table and tablet info
0925 20:00:26.160694 (+1191545us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 20:00:26.161540 (+ 846us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:26.161543 (+ 3us) write_op.cc:454] Released schema lock
0925 20:00:26.161763 (+ 220us) catalog_manager.cc:2220] Rolling back HMS table creation
0925 20:00:26.315596 (+153833us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 20:00:26.161466 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica 169b6de20560440b99cc08af549ce477 is not leader of this config: current role FOLLOWER)
0925 20:00:26.161479 (+ 13us) write_op.cc:489] Releasing partition, row and schema locks
0925 20:00:26.161486 (+ 7us) write_op.cc:454] Released schema lock
0925 20:00:26.161491 (+ 5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1214351,"Hive Metastore.run_cpu_time_us":530,"Hive Metastore.run_wall_time_us":130070,"child_traces":[["op",{}]]}
W20260925 20:00:26.318044 4176 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:26.377962 5029 connection.cc:573] server connection from 127.0.0.1:37952 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 20:00:26.378154 5065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "169b6de20560440b99cc08af549ce477" candidate_term: 5 candidate_status { last_received { term: 4 index: 85 } } ignore_live_leader: false dest_uuid: "da29ee550f82456fa7c5ce7ea916909f" is_pre_election: true
I20260925 20:00:26.378439 5065 raft_consensus.cc:2468] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 169b6de20560440b99cc08af549ce477 in term 4.
I20260925 20:00:26.379310 3871 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu with pid 4936
I20260925 20:00:26.379431 4175 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 169b6de20560440b99cc08af549ce477, da29ee550f82456fa7c5ce7ea916909f; no voters: 7f43a6a14ef547f2aeb22ce2b334383e
I20260925 20:00:26.379948 5280 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260925 20:00:26.380087 5280 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7f43a6a14ef547f2aeb22ce2b334383e)
I20260925 20:00:26.380182 5280 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 4 FOLLOWER]: Advancing to term 5
I20260925 20:00:26.382320 5280 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:26.383131 5280 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 5 election: Requested vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:26.385313 4176 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:26.392369 5248 raft_consensus.cc:493] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:26.392611 5248 raft_consensus.cc:515] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:26.393436 5248 leader_election.cc:290] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:26.394021 4665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5982cfa34e4e40d3894cbf46aab33715" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:26.394089 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5982cfa34e4e40d3894cbf46aab33715" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" is_pre_election: true
I20260925 20:00:26.394261 4665 raft_consensus.cc:2468] T 5982cfa34e4e40d3894cbf46aab33715 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:26.394331 4805 raft_consensus.cc:2468] T 5982cfa34e4e40d3894cbf46aab33715 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:26.394735 4455 leader_election.cc:304] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:26.395020 5248 raft_consensus.cc:2804] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:26.395153 5248 raft_consensus.cc:493] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:26.395252 5248 raft_consensus.cc:3060] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:26.399013 5248 raft_consensus.cc:515] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:26.399917 5248 leader_election.cc:290] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Requested vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:26.400542 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5982cfa34e4e40d3894cbf46aab33715" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
I20260925 20:00:26.400557 4665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5982cfa34e4e40d3894cbf46aab33715" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:26.400771 4665 raft_consensus.cc:3060] T 5982cfa34e4e40d3894cbf46aab33715 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:26.400774 4805 raft_consensus.cc:3060] T 5982cfa34e4e40d3894cbf46aab33715 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:26.403576 4805 raft_consensus.cc:2468] T 5982cfa34e4e40d3894cbf46aab33715 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
W20260925 20:00:26.404639 4175 connection.cc:573] client connection to 127.3.199.253:41897 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 20:00:26.404413 4665 raft_consensus.cc:2468] T 5982cfa34e4e40d3894cbf46aab33715 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
I20260925 20:00:26.406016 4457 leader_election.cc:304] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:26.406430 5248 raft_consensus.cc:2804] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:26.406622 5248 raft_consensus.cc:697] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 LEADER]: Becoming Leader. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Running, Role: LEADER
I20260925 20:00:26.406946 5248 consensus_queue.cc:237] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
W20260925 20:00:26.409332 4175 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:26.409520 4175 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:26.410085 5280 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260925 20:00:26.427290 5143 consensus_queue.cc:1048] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:26.435345 5143 consensus_queue.cc:1048] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 20:00:26.540194 5248 consensus_queue.cc:1048] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:26.546053 5248 consensus_queue.cc:1048] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:26.669307 5248 consensus_queue.cc:1048] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:26.674393 5187 consensus_queue.cc:1048] T 653a4121e8934471b99a9853b73617ab P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:26.983594 3871 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
/tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.253:41897
--webserver_interface=127.3.199.253
--webserver_port=41541
--builtin_ntp_servers=127.3.199.212:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 20:00:27.081743 5305 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:27.081977 5305 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:27.082784 5305 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:27.099606 4176 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260925 20:00:27.102797 4176 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:27.103027 4175 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:27.103210 4175 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:27.103554 5305 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260925 20:00:27.199867 5187 consensus_queue.cc:1048] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:27.216074 5314 consensus_queue.cc:1048] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:27.333847 5317 consensus_queue.cc:1048] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:27.340732 5317 consensus_queue.cc:1048] T 5982cfa34e4e40d3894cbf46aab33715 P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 20:00:27.401326 5304 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 20:00:27.401768 5304 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 20:00:27.401917 5304 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 20:00:27.411960 5304 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 20:00:27.412124 5304 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 20:00:27.412264 5304 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 20:00:27.412387 5304 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 20:00:27.425240 5304 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.199.212:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.253:41897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.199.253
--webserver_port=41541
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 25 Sep 2026 19:43:21 UTC on bdcb31816ec0
build id 13997
ASAN enabled
I20260925 20:00:27.427152 5304 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 20:00:27.429224 5304 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 20:00:27.438733 5330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:27.438727 5331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:27.439677 5333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 20:00:27.440325 5304 server_base.cc:1061] running on GCE node
I20260925 20:00:27.441256 5304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 20:00:27.442868 5304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 20:00:27.444116 5304 hybrid_clock.cc:648] HybridClock initialized: now 1790366427444049 us; error 59 us; skew 500 ppm
I20260925 20:00:27.444613 5304 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 20:00:27.447185 5304 webserver.cc:533] Webserver started at http://127.3.199.253:41541/ using document root <none> and password file <none>
I20260925 20:00:27.447983 5304 fs_manager.cc:362] Metadata directory not provided
I20260925 20:00:27.448122 5304 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 20:00:27.453843 5304 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260925 20:00:27.456943 5339 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:27.458534 5304 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 20:00:27.458920 5304 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data,/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
uuid: "da29ee550f82456fa7c5ce7ea916909f"
format_stamp: "Formatted at 2026-09-25 20:00:17 on dist-test-slave-9gwk"
I20260925 20:00:27.459684 5304 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 20:00:27.482122 5304 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 20:00:27.483153 5304 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 20:00:27.483505 5304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 20:00:27.510987 5304 rpc_server.cc:307] RPC server started. Bound to: 127.3.199.253:41897
I20260925 20:00:27.511005 5391 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.199.253:41897 every 8 connection(s)
I20260925 20:00:27.513453 5304 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
I20260925 20:00:27.518232 3871 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu as pid 5304
I20260925 20:00:27.521974 5392 sys_catalog.cc:263] Verifying existing consensus state
I20260925 20:00:27.527215 5392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Bootstrap starting.
I20260925 20:00:27.546304 5392 log.cc:826] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Log is configured to *not* fsync() on all Append() calls
I20260925 20:00:27.611685 5392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Bootstrap replayed 1/1 log segments. Stats: ops{read=85 overwritten=0 applied=85 ignored=0} inserts{seen=52 ignored=0} mutations{seen=62 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 20:00:27.612380 5392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Bootstrap complete.
I20260925 20:00:27.618139 5392 raft_consensus.cc:359] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:27.618937 5392 raft_consensus.cc:740] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: da29ee550f82456fa7c5ce7ea916909f, State: Initialized, Role: FOLLOWER
I20260925 20:00:27.619745 5392 consensus_queue.cc:260] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 85, Last appended: 4.85, Last appended by leader: 85, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:27.620821 5396 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:27.621184 5396 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:27.622063 5392 sys_catalog.cc:565] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 20:00:27.655026 5410 catalog_manager.cc:1308] Loaded cluster ID: 7efb83138ca94380bf2d8f0f2f91c381
I20260925 20:00:27.655160 5410 catalog_manager.cc:1601] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: loading cluster ID for follower catalog manager: success
I20260925 20:00:27.656349 4573 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:27.660231 5410 catalog_manager.cc:1623] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: acquiring CA information for follower catalog manager: success
I20260925 20:00:27.661599 5356 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" instance_seqno: 1790366418849966) as {username='slave'} at 127.3.199.193:46309; Asking this server to re-register.
I20260925 20:00:27.662818 4573 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:27.664027 5410 catalog_manager.cc:1651] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 20:00:27.665688 5356 ts_manager.cc:194] Registered new tserver with Master: 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:27.678126 5304 master_runner.cc:187] Error getting master registration for 127.3.199.252:37957: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111),
I20260925 20:00:27.682247 5415 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:27.682538 5304 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:00:27.682466 5415 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:27.683341 5415 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:27.690896 4176 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:27.694347 5366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "169b6de20560440b99cc08af549ce477" candidate_term: 6 candidate_status { last_received { term: 4 index: 85 } } ignore_live_leader: false dest_uuid: "da29ee550f82456fa7c5ce7ea916909f" is_pre_election: true
I20260925 20:00:27.695027 5366 raft_consensus.cc:2468] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 169b6de20560440b99cc08af549ce477 in term 4.
I20260925 20:00:27.695647 4175 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 169b6de20560440b99cc08af549ce477, da29ee550f82456fa7c5ce7ea916909f; no voters: 7f43a6a14ef547f2aeb22ce2b334383e
I20260925 20:00:27.696033 5415 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260925 20:00:27.696182 5415 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:27.696303 5415 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 5 FOLLOWER]: Advancing to term 6
I20260925 20:00:27.699489 5415 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:27.700747 5415 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 6 election: Requested vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:27.701433 5366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "169b6de20560440b99cc08af549ce477" candidate_term: 6 candidate_status { last_received { term: 4 index: 85 } } ignore_live_leader: false dest_uuid: "da29ee550f82456fa7c5ce7ea916909f"
I20260925 20:00:27.701666 5366 raft_consensus.cc:3060] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 4 FOLLOWER]: Advancing to term 6
W20260925 20:00:27.703677 4176 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:27.707057 5366 raft_consensus.cc:2468] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 169b6de20560440b99cc08af549ce477 in term 6.
I20260925 20:00:27.707628 4175 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 169b6de20560440b99cc08af549ce477, da29ee550f82456fa7c5ce7ea916909f; no voters: 7f43a6a14ef547f2aeb22ce2b334383e
I20260925 20:00:27.708024 5415 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 6 FOLLOWER]: Leader election won for term 6
I20260925 20:00:27.708587 5415 raft_consensus.cc:697] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 6 LEADER]: Becoming Leader. State: Replica: 169b6de20560440b99cc08af549ce477, State: Running, Role: LEADER
I20260925 20:00:27.708942 5415 consensus_queue.cc:237] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 85, Committed index: 85, Last appended: 4.85, Last appended by leader: 85, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:27.710302 5419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 169b6de20560440b99cc08af549ce477. Latest consensus state: current_term: 6 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:27.710618 5419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:27.711308 5421 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260925 20:00:27.714689 5421 catalog_manager.cc:692] Loaded metadata for table default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330]
I20260925 20:00:27.715188 5421 catalog_manager.cc:692] Loaded metadata for table default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf]
I20260925 20:00:27.715620 5421 catalog_manager.cc:692] Loaded metadata for table default.table_703616ef44ba43f9afb9c4df73eb372c [id=25f6e1ade43c4519bed519037ee8f761]
I20260925 20:00:27.716076 5421 catalog_manager.cc:692] Loaded metadata for table default.table_667eace0e712438bbd1211f7e3e665f8 [id=439b3d65790141d1abfbca4788836f80]
I20260925 20:00:27.716572 5421 catalog_manager.cc:692] Loaded metadata for table default.table_127c3d62fd1c4394bcd55bc6fc18bc5d [id=45cbddfbaced49a989c1fb61a644a713]
I20260925 20:00:27.717018 5421 catalog_manager.cc:692] Loaded metadata for table default.table_2ebd7ecb76b44d13a6847408f7fd7d17 [id=4b80e3afa6954e2ea8b6d1b5ec79a9d1]
I20260925 20:00:27.717417 5421 catalog_manager.cc:692] Loaded metadata for table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]
I20260925 20:00:27.717809 5421 catalog_manager.cc:692] Loaded metadata for table default.table_c78f9cc1d7e74b51b29858c7d3248fb8 [id=7ead82097c49448f85b1dbef027be066]
I20260925 20:00:27.718182 5421 catalog_manager.cc:692] Loaded metadata for table default.table_098b200909f842c88a38f11192f4dc25 [id=86f1c8a16cc14ef9b72c95060313b3ad]
I20260925 20:00:27.718607 5421 catalog_manager.cc:692] Loaded metadata for table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]
I20260925 20:00:27.719033 5421 catalog_manager.cc:692] Loaded metadata for table default.table_5b8386185e8e4fcd99110bd5137a578c [id=a385e0b7db6a49e8b1baf34a5fdd3473]
I20260925 20:00:27.719434 5421 catalog_manager.cc:692] Loaded metadata for table default.table_7722a249d10f4616ba5b5bb2fc149c18 [id=a8f568d7faab469ba7ea8a2a38bfd638]
I20260925 20:00:27.719820 5421 catalog_manager.cc:692] Loaded metadata for table default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d]
I20260925 20:00:27.720214 5421 catalog_manager.cc:692] Loaded metadata for table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]
I20260925 20:00:27.720831 5421 catalog_manager.cc:692] Loaded metadata for table default.table_08807903f9884f2a9707d4066e7f2f52 [id=f6fa165ad5e7471589cb63403548ab2c]
I20260925 20:00:27.722990 5421 tablet_loader.cc:96] loaded metadata for tablet 0827e2c7eff041369a94c31abdf9b093 (table default.table_127c3d62fd1c4394bcd55bc6fc18bc5d [id=45cbddfbaced49a989c1fb61a644a713])
I20260925 20:00:27.723424 5421 tablet_loader.cc:96] loaded metadata for tablet 0cd27940acbb43ef9d75591871c00481 (table default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330])
I20260925 20:00:27.724035 5421 tablet_loader.cc:96] loaded metadata for tablet 2ddb9fbd0aeb4c4a985508ac67c93227 (table default.table_7722a249d10f4616ba5b5bb2fc149c18 [id=a8f568d7faab469ba7ea8a2a38bfd638])
I20260925 20:00:27.724490 5421 tablet_loader.cc:96] loaded metadata for tablet 44f42691903d4e2bb3f24ecb75a3f86e (table default.table_08807903f9884f2a9707d4066e7f2f52 [id=f6fa165ad5e7471589cb63403548ab2c])
I20260925 20:00:27.725098 5421 tablet_loader.cc:96] loaded metadata for tablet 5982cfa34e4e40d3894cbf46aab33715 (table default.table_2ebd7ecb76b44d13a6847408f7fd7d17 [id=4b80e3afa6954e2ea8b6d1b5ec79a9d1])
I20260925 20:00:27.725492 5421 tablet_loader.cc:96] loaded metadata for tablet 59a14370409146a1847c5b76755e3feb (table default.table_c78f9cc1d7e74b51b29858c7d3248fb8 [id=7ead82097c49448f85b1dbef027be066])
I20260925 20:00:27.726303 5421 tablet_loader.cc:96] loaded metadata for tablet 653a4121e8934471b99a9853b73617ab (table default.table_5b8386185e8e4fcd99110bd5137a578c [id=a385e0b7db6a49e8b1baf34a5fdd3473])
I20260925 20:00:27.726675 5421 tablet_loader.cc:96] loaded metadata for tablet 72c3c64108f348a5ab893e6954d72d77 (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812])
I20260925 20:00:27.727303 5421 tablet_loader.cc:96] loaded metadata for tablet 8d0bc4cceef042389ad99a3515aa0f43 (table default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d])
I20260925 20:00:27.727669 5421 tablet_loader.cc:96] loaded metadata for tablet 983d375d3fda4e63ab93b6cde278e187 (table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7])
I20260925 20:00:27.728547 5421 tablet_loader.cc:96] loaded metadata for tablet b59f74f459c542a88e8bef5f1c51357d (table default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf])
I20260925 20:00:27.728937 5421 tablet_loader.cc:96] loaded metadata for tablet b65400989b944ca988d6f26838277336 (table default.table_667eace0e712438bbd1211f7e3e665f8 [id=439b3d65790141d1abfbca4788836f80])
I20260925 20:00:27.729828 5421 tablet_loader.cc:96] loaded metadata for tablet e0648fcd521e4db19359c743447246b7 (table default.table_703616ef44ba43f9afb9c4df73eb372c [id=25f6e1ade43c4519bed519037ee8f761])
I20260925 20:00:27.730197 5421 tablet_loader.cc:96] loaded metadata for tablet ee843d55757347ef9942dffd917c0ff8 (table default.table_098b200909f842c88a38f11192f4dc25 [id=86f1c8a16cc14ef9b72c95060313b3ad])
I20260925 20:00:27.730645 5421 tablet_loader.cc:96] loaded metadata for tablet f4b5baba575d4538b807a69eff0d90dd (table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22])
I20260925 20:00:27.731601 5421 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260925 20:00:27.733210 5421 catalog_manager.cc:1308] Loaded cluster ID: 7efb83138ca94380bf2d8f0f2f91c381
I20260925 20:00:27.733317 5421 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260925 20:00:27.735188 5421 catalog_manager.cc:1553] Loading token signing keys...
I20260925 20:00:27.736622 5421 catalog_manager.cc:6230] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Loaded TSK: 0
I20260925 20:00:27.737128 5421 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260925 20:00:27.738444 5421 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260925 20:00:27.739792 5421 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: 20
I20260925 20:00:27.755404 4189 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_30a6c0d501b04bfc882cec2f43c06f83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:27.755823 4187 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_9f19364a66794406806691ab3a701b72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:27.756708 4180 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_2ebd7ecb76b44d13a6847408f7fd7d17" } modify_external_catalogs: true
I20260925 20:00:27.756994 4183 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_667eace0e712438bbd1211f7e3e665f8" } modify_external_catalogs: true
I20260925 20:00:27.757032 4180 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_2ebd7ecb76b44d13a6847408f7fd7d17" } modify_external_catalogs: true
I20260925 20:00:27.757198 4183 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_667eace0e712438bbd1211f7e3e665f8" } modify_external_catalogs: true
I20260925 20:00:27.757469 4188 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_127c3d62fd1c4394bcd55bc6fc18bc5d" } modify_external_catalogs: true
I20260925 20:00:27.757437 4184 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_4627e927cd124d049b0fdc22eb15c554"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:27.757664 4188 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48108:
table { table_name: "default.Table_127c3d62fd1c4394bcd55bc6fc18bc5d" } modify_external_catalogs: true
I20260925 20:00:27.757037 4185 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_aae983110c484d68a3a085faa1688c5e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:27.760316 4181 master_service.cc:960] ReplaceTablet: received request to replace tablet 0cd27940acbb43ef9d75591871c00481 from {username='slave'} at 127.0.0.1:48108
F20260925 20:00:27.763451 4403 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 21, previous 21; dumping first 2 out of 13 received events: NotificationEvent(eventId=21, eventTime=1790366425, eventType=DROP_TABLE, dbName=default, tableName=table_30a6c0d501b04bfc882cec2f43c06f83, message={"server":"","servicePrincipal":"","db":"default","table":"table_30a6c0d501b04bfc882cec2f43c06f83","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_30a6c0d501b04bfc882cec2f43c06f83\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790366424},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/warehouse/table_30a6c0d501b04bfc882cec2f43c06f83\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_30a6c0d501b04bfc882cec2f43c06f83\",\"kudu.table_id\":\"44287e882a4040dcb79d824c08808dd4\",\"transient_lastDdlTime\":\"1790366424\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"7efb83138ca94380bf2d8f0f2f91c381\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790366425}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=21, eventTime=1790366425, eventType=CREATE_TABLE, dbName=default, tableName=table_127c3d62fd1c4394bcd55bc6fc18bc5d, message={"server":"","servicePrincipal":"","db":"default","table":"table_127c3d62fd1c4394bcd55bc6fc18bc5d","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_127c3d62fd1c4394bcd55bc6fc18bc5d\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790366425},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/warehouse/table_127c3d62fd1c4394bcd55bc6fc18bc5d\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_127c3d62fd1c4394bcd55bc6fc18bc5d\",\"kudu.table_id\":\"45cbddfbaced49a989c1fb61a644a713\",\"transient_lastDdlTime\":\"1790366425\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"7efb83138ca94380bf2d8f0f2f91c381\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790366425,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260925 20:00:27.763948 4182 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_d4b20390d75f45c89df2fe5d8498b337"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:27.765209 4186 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48108:
name: "default.Table_ef69b46f2522492baaac95731da2bbcb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:27.767717 5366 raft_consensus.cc:1275] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Refusing update from remote peer 169b6de20560440b99cc08af549ce477: Log matching property violated. Preceding OpId in replica: term: 4 index: 85. Preceding OpId from leader: term: 6 index: 87. (index mismatch)
@ 0x7fc4f3696d5d google::LogMessage::Fail() at ??:0
I20260925 20:00:27.768635 5415 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 86, Last known committed idx: 85, Time since last communication: 0.000s
@ 0x7fc4f369ab23 google::LogMessage::SendToLog() at ??:0
@ 0x7fc4f369675c google::LogMessage::Flush() at ??:0
W20260925 20:00:27.771668 4176 consensus_peers.cc:597] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 -> Peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Couldn't send request to peer 7f43a6a14ef547f2aeb22ce2b334383e. Status: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7fc4f36969a1 google::LogMessage::~LogMessage() at ??:0
@ 0x7fc50a3952f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7fc50a393ba1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7fc50a39b81c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7fc50a39b699 std::_Function_handler<>::_M_invoke() at ??:0
I20260925 20:00:27.801990 5396 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 169b6de20560440b99cc08af549ce477. Latest consensus state: current_term: 6 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:27.802350 5396 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: FOLLOWER
@ 0xaaef9d std::function<>::operator()() at ??:?
I20260925 20:00:27.806689 5415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:27.806967 5415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:27.808656 4181 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Sending DeleteTablet for 3 replicas of tablet 0cd27940acbb43ef9d75591871c00481
@ 0x7fc4f63e6079 kudu::Thread::SuperviseThread() at ??:0
I20260925 20:00:27.809356 4181 catalog_manager.cc:6934] ReplaceTablet: tablet 0cd27940acbb43ef9d75591871c00481 deleted and replaced by tablet 83c0e7bd6ef040b9af169d9677137836
I20260925 20:00:27.809690 4784 tablet_service.cc:1558] Processing DeleteTablet for tablet 0cd27940acbb43ef9d75591871c00481 with delete_type TABLET_DATA_DELETED (replaced by tablet 83c0e7bd6ef040b9af169d9677137836) from {username='slave'} at 127.0.0.1:54514
I20260925 20:00:27.809720 4181 master_service.cc:960] ReplaceTablet: received request to replace tablet 72c3c64108f348a5ab893e6954d72d77 from {username='slave'} at 127.0.0.1:48108
I20260925 20:00:27.810133 4506 tablet_service.cc:1558] Processing DeleteTablet for tablet 0cd27940acbb43ef9d75591871c00481 with delete_type TABLET_DATA_DELETED (replaced by tablet 83c0e7bd6ef040b9af169d9677137836) from {username='slave'} at 127.0.0.1:54948
I20260925 20:00:27.810539 4644 tablet_service.cc:1558] Processing DeleteTablet for tablet 0cd27940acbb43ef9d75591871c00481 with delete_type TABLET_DATA_DELETED (replaced by tablet 83c0e7bd6ef040b9af169d9677137836) from {username='slave'} at 127.0.0.1:57378
@ 0x7fc4f9abc6db start_thread at ??:0
@ 0x7fc4f254361f clone at ??:0
I20260925 20:00:27.815357 5434 tablet_replica.cc:333] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:27.815644 5434 raft_consensus.cc:2243] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:27.816126 5434 raft_consensus.cc:2272] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:27.816875 5434 ts_tablet_manager.cc:1916] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:27.820797 5433 tablet_replica.cc:333] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:27.821143 5433 raft_consensus.cc:2243] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:27.821391 5433 raft_consensus.cc:2272] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:27.822227 5433 ts_tablet_manager.cc:1916] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:27.827162 5434 ts_tablet_manager.cc:1929] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:27.827299 5434 log.cc:1199] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/0cd27940acbb43ef9d75591871c00481
I20260925 20:00:27.827768 5434 ts_tablet_manager.cc:1950] T 0cd27940acbb43ef9d75591871c00481 P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:27.830144 5433 ts_tablet_manager.cc:1929] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:27.830219 5432 tablet_replica.cc:333] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:27.830281 5433 log.cc:1199] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/0cd27940acbb43ef9d75591871c00481
I20260925 20:00:27.830471 5432 raft_consensus.cc:2243] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:27.830691 5432 raft_consensus.cc:2272] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:27.830770 5433 ts_tablet_manager.cc:1950] T 0cd27940acbb43ef9d75591871c00481 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:27.831497 5432 ts_tablet_manager.cc:1916] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:27.842788 5432 ts_tablet_manager.cc:1929] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:27.842671 5396 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "169b6de20560440b99cc08af549ce477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:27.842957 5432 log.cc:1199] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/0cd27940acbb43ef9d75591871c00481
I20260925 20:00:27.842962 5396 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:27.843453 5432 ts_tablet_manager.cc:1950] T 0cd27940acbb43ef9d75591871c00481 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:27.857328 4714 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:27.859679 5356 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" instance_seqno: 1790366419525613) as {username='slave'} at 127.3.199.194:51371; Asking this server to re-register.
I20260925 20:00:27.860596 4714 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:27.862818 5356 ts_manager.cc:194] Registered new tserver with Master: 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:27.919277 4852 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:27.921543 5356 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" instance_seqno: 1790366420065491) as {username='slave'} at 127.3.199.195:47857; Asking this server to re-register.
I20260925 20:00:27.922314 4852 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:27.924136 5356 ts_manager.cc:194] Registered new tserver with Master: dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
W20260925 20:00:28.124752 4600 connection.cc:573] server connection from 127.0.0.1:57378 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:28.125006 4862 connection.cc:573] client connection to 127.3.199.254:41289 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:28.125041 4602 connection.cc:573] client connection to 127.3.199.254:41289 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:28.125401 4740 connection.cc:573] client connection to 127.3.199.254:41289 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:28.125792 4862 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.3.199.254:41289)
W20260925 20:00:28.126075 4862 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.199.254:41289)
W20260925 20:00:28.126499 4862 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.3.199.254:41289)
W20260925 20:00:28.126714 4862 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.199.254:41289)
W20260925 20:00:28.128671 5341 connection.cc:573] server connection from 127.0.0.1:44632 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:28.129117 4457 connection.cc:573] server connection from 127.0.0.1:54948 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:28.129422 4738 connection.cc:573] server connection from 127.0.0.1:54514 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:28.129580 4456 connection.cc:573] client connection to 127.3.199.254:41289 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 20:00:28.308245 5396 raft_consensus.cc:493] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 169b6de20560440b99cc08af549ce477)
I20260925 20:00:28.308565 5396 raft_consensus.cc:515] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:28.310683 5396 leader_election.cc:290] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:28.312153 5343 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:28.314116 5343 leader_election.cc:336] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:28.314435 5342 leader_election.cc:336] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289): Network error: Client connection negotiation failed: client connection to 127.3.199.254:41289: connect: Connection refused (error 111)
I20260925 20:00:28.314575 5342 leader_election.cc:304] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da29ee550f82456fa7c5ce7ea916909f; no voters: 169b6de20560440b99cc08af549ce477, 7f43a6a14ef547f2aeb22ce2b334383e
I20260925 20:00:28.314970 5396 raft_consensus.cc:2749] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260925 20:00:28.900187 5454 raft_consensus.cc:493] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 169b6de20560440b99cc08af549ce477)
I20260925 20:00:28.900458 5454 raft_consensus.cc:515] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:28.901680 5454 leader_election.cc:290] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:28.904407 5343 leader_election.cc:336] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:28.904627 5342 leader_election.cc:336] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289): Network error: Client connection negotiation failed: client connection to 127.3.199.254:41289: connect: Connection refused (error 111)
I20260925 20:00:28.904778 5342 leader_election.cc:304] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da29ee550f82456fa7c5ce7ea916909f; no voters: 169b6de20560440b99cc08af549ce477, 7f43a6a14ef547f2aeb22ce2b334383e
I20260925 20:00:28.905124 5454 raft_consensus.cc:2749] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260925 20:00:29.931308 5463 raft_consensus.cc:493] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 169b6de20560440b99cc08af549ce477)
I20260925 20:00:29.931535 5463 raft_consensus.cc:515] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:29.932194 5463 leader_election.cc:290] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:29.934739 5343 leader_election.cc:336] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:29.934962 5342 leader_election.cc:336] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289): Network error: Client connection negotiation failed: client connection to 127.3.199.254:41289: connect: Connection refused (error 111)
I20260925 20:00:29.935105 5342 leader_election.cc:304] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da29ee550f82456fa7c5ce7ea916909f; no voters: 169b6de20560440b99cc08af549ce477, 7f43a6a14ef547f2aeb22ce2b334383e
I20260925 20:00:29.935374 5463 raft_consensus.cc:2749] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260925 20:00:30.481424 5471 raft_consensus.cc:493] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 169b6de20560440b99cc08af549ce477)
I20260925 20:00:30.481669 5471 raft_consensus.cc:515] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:30.482470 5471 leader_election.cc:290] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:30.485100 5342 leader_election.cc:336] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289): Network error: Client connection negotiation failed: client connection to 127.3.199.254:41289: connect: Connection refused (error 111)
W20260925 20:00:30.485323 5343 leader_election.cc:336] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:30.485445 5343 leader_election.cc:304] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da29ee550f82456fa7c5ce7ea916909f; no voters: 169b6de20560440b99cc08af549ce477, 7f43a6a14ef547f2aeb22ce2b334383e
I20260925 20:00:30.485674 5471 raft_consensus.cc:2749] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260925 20:00:30.750514 3871 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
/tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.254:41289
--webserver_interface=127.3.199.254
--webserver_port=32859
--builtin_ntp_servers=127.3.199.212:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260925 20:00:31.120479 5477 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 20:00:31.120826 5477 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 20:00:31.120908 5477 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 20:00:31.129781 5477 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 20:00:31.129881 5477 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 20:00:31.129936 5477 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 20:00:31.129976 5477 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 20:00:31.140204 5477 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.199.212:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.254:41289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.199.254
--webserver_port=32859
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 25 Sep 2026 19:43:21 UTC on bdcb31816ec0
build id 13997
ASAN enabled
I20260925 20:00:31.141865 5477 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 20:00:31.143612 5477 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 20:00:31.152580 5483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:31.152892 5486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:31.152554 5484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 20:00:31.154462 5477 server_base.cc:1061] running on GCE node
I20260925 20:00:31.155246 5477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 20:00:31.156965 5477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 20:00:31.158238 5477 hybrid_clock.cc:648] HybridClock initialized: now 1790366431158179 us; error 78 us; skew 500 ppm
I20260925 20:00:31.158670 5477 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 20:00:31.161226 5477 webserver.cc:533] Webserver started at http://127.3.199.254:32859/ using document root <none> and password file <none>
I20260925 20:00:31.162042 5477 fs_manager.cc:362] Metadata directory not provided
I20260925 20:00:31.162196 5477 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 20:00:31.169080 5477 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260925 20:00:31.172225 5494 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:31.174088 5477 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 20:00:31.174324 5477 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data,/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
uuid: "169b6de20560440b99cc08af549ce477"
format_stamp: "Formatted at 2026-09-25 20:00:16 on dist-test-slave-9gwk"
I20260925 20:00:31.175319 5477 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 20:00:31.194667 5477 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 20:00:31.195851 5477 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 20:00:31.196158 5477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 20:00:31.218271 5477 rpc_server.cc:307] RPC server started. Bound to: 127.3.199.254:41289
I20260925 20:00:31.218298 5545 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.199.254:41289 every 8 connection(s)
I20260925 20:00:31.220360 5477 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data/info.pb
I20260925 20:00:31.221642 3871 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu as pid 5477
I20260925 20:00:31.229077 5546 sys_catalog.cc:263] Verifying existing consensus state
I20260925 20:00:31.233094 5546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Bootstrap starting.
I20260925 20:00:31.242384 5549 raft_consensus.cc:493] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 169b6de20560440b99cc08af549ce477)
I20260925 20:00:31.242601 5549 raft_consensus.cc:515] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:31.243734 5549 leader_election.cc:290] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:31.247928 5343 leader_election.cc:336] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:31.257714 5546 log.cc:826] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Log is configured to *not* fsync() on all Append() calls
I20260925 20:00:31.329293 5546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Bootstrap replayed 1/1 log segments. Stats: ops{read=87 overwritten=0 applied=87 ignored=0} inserts{seen=53 ignored=0} mutations{seen=63 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 20:00:31.329900 5546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Bootstrap complete.
I20260925 20:00:31.335424 5546 raft_consensus.cc:359] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:31.336393 5546 raft_consensus.cc:740] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 169b6de20560440b99cc08af549ce477, State: Initialized, Role: FOLLOWER
I20260925 20:00:31.337215 5546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 87, Last appended: 6.87, Last appended by leader: 87, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:31.338611 5554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:31.339005 5554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:31.339767 5546 sys_catalog.cc:565] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 20:00:31.339862 5521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da29ee550f82456fa7c5ce7ea916909f" candidate_term: 7 candidate_status { last_received { term: 6 index: 87 } } ignore_live_leader: false dest_uuid: "169b6de20560440b99cc08af549ce477" is_pre_election: true
I20260925 20:00:31.340528 5521 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da29ee550f82456fa7c5ce7ea916909f in term 6.
I20260925 20:00:31.341171 5342 leader_election.cc:304] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 169b6de20560440b99cc08af549ce477, da29ee550f82456fa7c5ce7ea916909f; no voters: 7f43a6a14ef547f2aeb22ce2b334383e
I20260925 20:00:31.341604 5549 raft_consensus.cc:2804] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260925 20:00:31.341758 5549 raft_consensus.cc:493] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 169b6de20560440b99cc08af549ce477)
I20260925 20:00:31.341883 5549 raft_consensus.cc:3060] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 6 FOLLOWER]: Advancing to term 7
I20260925 20:00:31.345402 5549 raft_consensus.cc:515] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:31.347855 5549 leader_election.cc:290] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 election: Requested vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:31.348485 5521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da29ee550f82456fa7c5ce7ea916909f" candidate_term: 7 candidate_status { last_received { term: 6 index: 87 } } ignore_live_leader: false dest_uuid: "169b6de20560440b99cc08af549ce477"
I20260925 20:00:31.348760 5521 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 6 FOLLOWER]: Advancing to term 7
W20260925 20:00:31.349866 5343 leader_election.cc:336] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:31.355808 5521 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da29ee550f82456fa7c5ce7ea916909f in term 7.
I20260925 20:00:31.356694 5342 leader_election.cc:304] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 169b6de20560440b99cc08af549ce477, da29ee550f82456fa7c5ce7ea916909f; no voters: 7f43a6a14ef547f2aeb22ce2b334383e
I20260925 20:00:31.357167 5549 raft_consensus.cc:2804] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 7 FOLLOWER]: Leader election won for term 7
I20260925 20:00:31.357723 5549 raft_consensus.cc:697] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 7 LEADER]: Becoming Leader. State: Replica: da29ee550f82456fa7c5ce7ea916909f, State: Running, Role: LEADER
I20260925 20:00:31.358311 5549 consensus_queue.cc:237] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 87, Committed index: 87, Last appended: 6.87, Last appended by leader: 87, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:31.360416 5571 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: New leader da29ee550f82456fa7c5ce7ea916909f. Latest consensus state: current_term: 7 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:31.360788 5571 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:31.361552 5573 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260925 20:00:31.367028 5573 catalog_manager.cc:692] Loaded metadata for table default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330]
I20260925 20:00:31.371634 5568 catalog_manager.cc:1308] Loaded cluster ID: 7efb83138ca94380bf2d8f0f2f91c381
I20260925 20:00:31.371778 5568 catalog_manager.cc:1601] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: loading cluster ID for follower catalog manager: success
I20260925 20:00:31.383229 5573 catalog_manager.cc:692] Loaded metadata for table default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf]
I20260925 20:00:31.383874 5573 catalog_manager.cc:692] Loaded metadata for table default.table_703616ef44ba43f9afb9c4df73eb372c [id=25f6e1ade43c4519bed519037ee8f761]
I20260925 20:00:31.384373 5573 catalog_manager.cc:692] Loaded metadata for table default.table_667eace0e712438bbd1211f7e3e665f8 [id=439b3d65790141d1abfbca4788836f80]
I20260925 20:00:31.384879 5573 catalog_manager.cc:692] Loaded metadata for table default.table_127c3d62fd1c4394bcd55bc6fc18bc5d [id=45cbddfbaced49a989c1fb61a644a713]
I20260925 20:00:31.385361 5573 catalog_manager.cc:692] Loaded metadata for table default.table_2ebd7ecb76b44d13a6847408f7fd7d17 [id=4b80e3afa6954e2ea8b6d1b5ec79a9d1]
I20260925 20:00:31.385813 5573 catalog_manager.cc:692] Loaded metadata for table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]
I20260925 20:00:31.386257 5573 catalog_manager.cc:692] Loaded metadata for table default.table_c78f9cc1d7e74b51b29858c7d3248fb8 [id=7ead82097c49448f85b1dbef027be066]
I20260925 20:00:31.386677 5573 catalog_manager.cc:692] Loaded metadata for table default.table_098b200909f842c88a38f11192f4dc25 [id=86f1c8a16cc14ef9b72c95060313b3ad]
I20260925 20:00:31.387125 5573 catalog_manager.cc:692] Loaded metadata for table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]
I20260925 20:00:31.395104 5568 catalog_manager.cc:1623] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: acquiring CA information for follower catalog manager: success
I20260925 20:00:31.397974 5573 catalog_manager.cc:692] Loaded metadata for table default.table_5b8386185e8e4fcd99110bd5137a578c [id=a385e0b7db6a49e8b1baf34a5fdd3473]
I20260925 20:00:31.398572 5573 catalog_manager.cc:692] Loaded metadata for table default.table_7722a249d10f4616ba5b5bb2fc149c18 [id=a8f568d7faab469ba7ea8a2a38bfd638]
I20260925 20:00:31.399073 5573 catalog_manager.cc:692] Loaded metadata for table default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d]
I20260925 20:00:31.399523 5573 catalog_manager.cc:692] Loaded metadata for table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]
I20260925 20:00:31.400190 5573 catalog_manager.cc:692] Loaded metadata for table default.table_08807903f9884f2a9707d4066e7f2f52 [id=f6fa165ad5e7471589cb63403548ab2c]
I20260925 20:00:31.400386 5568 catalog_manager.cc:1651] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 20:00:31.402788 5573 tablet_loader.cc:96] loaded metadata for tablet 0827e2c7eff041369a94c31abdf9b093 (table default.table_127c3d62fd1c4394bcd55bc6fc18bc5d [id=45cbddfbaced49a989c1fb61a644a713])
I20260925 20:00:31.403805 5573 tablet_loader.cc:96] loaded metadata for tablet 2ddb9fbd0aeb4c4a985508ac67c93227 (table default.table_7722a249d10f4616ba5b5bb2fc149c18 [id=a8f568d7faab469ba7ea8a2a38bfd638])
I20260925 20:00:31.404219 5573 tablet_loader.cc:96] loaded metadata for tablet 44f42691903d4e2bb3f24ecb75a3f86e (table default.table_08807903f9884f2a9707d4066e7f2f52 [id=f6fa165ad5e7471589cb63403548ab2c])
I20260925 20:00:31.406562 5573 tablet_loader.cc:96] loaded metadata for tablet 5982cfa34e4e40d3894cbf46aab33715 (table default.table_2ebd7ecb76b44d13a6847408f7fd7d17 [id=4b80e3afa6954e2ea8b6d1b5ec79a9d1])
I20260925 20:00:31.407112 5573 tablet_loader.cc:96] loaded metadata for tablet 59a14370409146a1847c5b76755e3feb (table default.table_c78f9cc1d7e74b51b29858c7d3248fb8 [id=7ead82097c49448f85b1dbef027be066])
I20260925 20:00:31.408061 5573 tablet_loader.cc:96] loaded metadata for tablet 653a4121e8934471b99a9853b73617ab (table default.table_5b8386185e8e4fcd99110bd5137a578c [id=a385e0b7db6a49e8b1baf34a5fdd3473])
I20260925 20:00:31.408634 5573 tablet_loader.cc:96] loaded metadata for tablet 72c3c64108f348a5ab893e6954d72d77 (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812])
I20260925 20:00:31.409022 5573 tablet_loader.cc:96] loaded metadata for tablet 83c0e7bd6ef040b9af169d9677137836 (table default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330])
I20260925 20:00:31.409698 5573 tablet_loader.cc:96] loaded metadata for tablet 8d0bc4cceef042389ad99a3515aa0f43 (table default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d])
I20260925 20:00:31.409777 4713 heartbeater.cc:344] Connected to a master server at 127.3.199.254:41289
I20260925 20:00:31.410144 5573 tablet_loader.cc:96] loaded metadata for tablet 983d375d3fda4e63ab93b6cde278e187 (table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7])
I20260925 20:00:31.410796 5573 tablet_loader.cc:96] loaded metadata for tablet b59f74f459c542a88e8bef5f1c51357d (table default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf])
I20260925 20:00:31.411155 5573 tablet_loader.cc:96] loaded metadata for tablet b65400989b944ca988d6f26838277336 (table default.table_667eace0e712438bbd1211f7e3e665f8 [id=439b3d65790141d1abfbca4788836f80])
I20260925 20:00:31.412451 5573 tablet_loader.cc:96] loaded metadata for tablet e0648fcd521e4db19359c743447246b7 (table default.table_703616ef44ba43f9afb9c4df73eb372c [id=25f6e1ade43c4519bed519037ee8f761])
I20260925 20:00:31.414202 5511 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" instance_seqno: 1790366419525613) as {username='slave'} at 127.3.199.194:46435; Asking this server to re-register.
I20260925 20:00:31.420070 5573 tablet_loader.cc:96] loaded metadata for tablet ee843d55757347ef9942dffd917c0ff8 (table default.table_098b200909f842c88a38f11192f4dc25 [id=86f1c8a16cc14ef9b72c95060313b3ad])
I20260925 20:00:31.420583 5573 tablet_loader.cc:96] loaded metadata for tablet f4b5baba575d4538b807a69eff0d90dd (table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22])
I20260925 20:00:31.421231 5573 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260925 20:00:31.421311 4713 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:31.425654 5573 catalog_manager.cc:1308] Loaded cluster ID: 7efb83138ca94380bf2d8f0f2f91c381
I20260925 20:00:31.425769 5573 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260925 20:00:31.427146 5511 ts_manager.cc:194] Registered new tserver with Master: 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:31.428798 5573 catalog_manager.cc:1553] Loading token signing keys...
I20260925 20:00:31.430020 5573 catalog_manager.cc:6230] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Loaded TSK: 0
I20260925 20:00:31.431195 5573 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260925 20:00:31.432837 5573 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260925 20:00:31.434381 5573 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: 20
I20260925 20:00:31.430519 5477 master_runner.cc:187] Error getting master registration for 127.3.199.252:37957: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111),
I20260925 20:00:31.437922 5477 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:00:31.443728 4851 heartbeater.cc:344] Connected to a master server at 127.3.199.254:41289
I20260925 20:00:31.446019 5511 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" instance_seqno: 1790366420065491) as {username='slave'} at 127.3.199.195:56901; Asking this server to re-register.
I20260925 20:00:31.446712 4851 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:31.448455 5511 ts_manager.cc:194] Registered new tserver with Master: dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:31.516732 5354 master_service.cc:960] ReplaceTablet: received request to replace tablet f4b5baba575d4538b807a69eff0d90dd from {username='slave'} at 127.0.0.1:44610
I20260925 20:00:31.517059 5355 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44610:
name: "default.Table_799678007be3498da115944e990a1785"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:31.517408 5352 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44610:
table { table_name: "default.Table_cf7560049cf84f22b27dd94a6b4dbcef" } modify_external_catalogs: true
I20260925 20:00:31.517760 5352 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44610:
table { table_name: "default.Table_cf7560049cf84f22b27dd94a6b4dbcef" } modify_external_catalogs: true
I20260925 20:00:31.518148 5353 master_service.cc:960] ReplaceTablet: received request to replace tablet f4b5baba575d4538b807a69eff0d90dd from {username='slave'} at 127.0.0.1:44610
I20260925 20:00:31.519425 5350 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44610:
table { table_name: "default.Table_e72a162b7a554904a9ca0cceb5e6b273" } new_table_name: "default.Table_1073b55979e5429c83265f7d64835607" modify_external_catalogs: true
I20260925 20:00:31.519523 5347 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44610:
table { table_name: "default.Table_b74d63554f344195800ce065c2a3cddf" } new_table_name: "default.Table_1f3deb6e388f4abf83c97af376b6cf64" modify_external_catalogs: true
I20260925 20:00:31.516569 5356 master_service.cc:960] ReplaceTablet: received request to replace tablet 983d375d3fda4e63ab93b6cde278e187 from {username='slave'} at 127.0.0.1:44610
I20260925 20:00:31.520920 5351 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44610:
table { table_name: "default.Table_c78f9cc1d7e74b51b29858c7d3248fb8" } new_table_name: "default.Table_1f01c68bbe404907a4d5f4d12fb9ee26" modify_external_catalogs: true
I20260925 20:00:31.520174 5349 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44610:
name: "default.Table_f051ca2de9a64755b465c9b043ce4392"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:31.526708 5348 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44610:
table { table_name: "default.Table_08807903f9884f2a9707d4066e7f2f52" } new_table_name: "default.Table_6ea0804544d84afca71f1f239845be69" modify_external_catalogs: true
I20260925 20:00:31.536513 5521 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Refusing update from remote peer da29ee550f82456fa7c5ce7ea916909f: Log matching property violated. Preceding OpId in replica: term: 6 index: 87. Preceding OpId from leader: term: 7 index: 89. (index mismatch)
W20260925 20:00:31.536780 5343 consensus_peers.cc:597] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f -> Peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Couldn't send request to peer 7f43a6a14ef547f2aeb22ce2b334383e. Status: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 20:00:31.537508 5549 consensus_queue.cc:1048] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [LEADER]: Connected to new peer: Peer: permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 88, Last known committed idx: 87, Time since last communication: 0.000s
F20260925 20:00:31.544613 5409 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 21, previous 21; dumping first 2 out of 13 received events: NotificationEvent(eventId=21, eventTime=1790366425, eventType=DROP_TABLE, dbName=default, tableName=table_30a6c0d501b04bfc882cec2f43c06f83, message={"server":"","servicePrincipal":"","db":"default","table":"table_30a6c0d501b04bfc882cec2f43c06f83","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_30a6c0d501b04bfc882cec2f43c06f83\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790366424},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/warehouse/table_30a6c0d501b04bfc882cec2f43c06f83\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_30a6c0d501b04bfc882cec2f43c06f83\",\"kudu.table_id\":\"44287e882a4040dcb79d824c08808dd4\",\"transient_lastDdlTime\":\"1790366424\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"7efb83138ca94380bf2d8f0f2f91c381\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790366425}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=21, eventTime=1790366425, eventType=CREATE_TABLE, dbName=default, tableName=table_127c3d62fd1c4394bcd55bc6fc18bc5d, message={"server":"","servicePrincipal":"","db":"default","table":"table_127c3d62fd1c4394bcd55bc6fc18bc5d","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_127c3d62fd1c4394bcd55bc6fc18bc5d\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790366425},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/warehouse/table_127c3d62fd1c4394bcd55bc6fc18bc5d\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_127c3d62fd1c4394bcd55bc6fc18bc5d\",\"kudu.table_id\":\"45cbddfbaced49a989c1fb61a644a713\",\"transient_lastDdlTime\":\"1790366425\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"7efb83138ca94380bf2d8f0f2f91c381\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790366425,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f362ebfbd5d google::LogMessage::Fail() at ??:0
@ 0x7f362ebffb23 google::LogMessage::SendToLog() at ??:0
@ 0x7f362ebfb75c google::LogMessage::Flush() at ??:0
I20260925 20:00:31.551371 5554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da29ee550f82456fa7c5ce7ea916909f. Latest consensus state: current_term: 7 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:31.551698 5554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f362ebfb9a1 google::LogMessage::~LogMessage() at ??:0
I20260925 20:00:31.554517 5581 mvcc.cc:204] Tried to move back new op lower bound from 7333340903562096640 to 7333340902848233472. Current Snapshot: MvccSnapshot[applied={T|T < 7333340903562096640}]
I20260925 20:00:31.558455 5554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:31.558807 5554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:31.564061 5354 catalog_manager.cc:6133] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Sending DeleteTablet for 3 replicas of tablet f4b5baba575d4538b807a69eff0d90dd
I20260925 20:00:31.566180 5354 catalog_manager.cc:6934] ReplaceTablet: tablet f4b5baba575d4538b807a69eff0d90dd deleted and replaced by tablet dbdd993621c4410386d7914283809577
I20260925 20:00:31.575093 5582 mvcc.cc:204] Tried to move back new op lower bound from 7333340903566323712 to 7333340902848233472. Current Snapshot: MvccSnapshot[applied={T|T < 7333340903566323712}]
I20260925 20:00:31.583716 5356 catalog_manager.cc:6133] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Sending DeleteTablet for 3 replicas of tablet 983d375d3fda4e63ab93b6cde278e187
I20260925 20:00:31.584612 5356 catalog_manager.cc:6934] ReplaceTablet: tablet 983d375d3fda4e63ab93b6cde278e187 deleted and replaced by tablet b321f36472cc42babf1ec3f64ab990ed
@ 0x7f36458fa2f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260925 20:00:31.593376 4506 tablet_service.cc:1558] Processing DeleteTablet for tablet f4b5baba575d4538b807a69eff0d90dd with delete_type TABLET_DATA_DELETED (replaced by tablet dbdd993621c4410386d7914283809577) from {username='slave'} at 127.0.0.1:50550
I20260925 20:00:31.594013 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet 983d375d3fda4e63ab93b6cde278e187 with delete_type TABLET_DATA_DELETED (replaced by tablet b321f36472cc42babf1ec3f64ab990ed) from {username='slave'} at 127.0.0.1:50550
@ 0x7f36458f8ba1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260925 20:00:31.599282 4644 tablet_service.cc:1558] Processing DeleteTablet for tablet f4b5baba575d4538b807a69eff0d90dd with delete_type TABLET_DATA_DELETED (replaced by tablet dbdd993621c4410386d7914283809577) from {username='slave'} at 127.0.0.1:43692
I20260925 20:00:31.600092 5594 tablet_replica.cc:333] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:31.600208 4644 tablet_service.cc:1558] Processing DeleteTablet for tablet 983d375d3fda4e63ab93b6cde278e187 with delete_type TABLET_DATA_DELETED (replaced by tablet b321f36472cc42babf1ec3f64ab990ed) from {username='slave'} at 127.0.0.1:43692
I20260925 20:00:31.601361 5353 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44610:
name: "default.Table_6796b08b82a249d8b499df979ebeb52a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:31.607832 5356 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44610:
table { table_name: "default.Table_098b200909f842c88a38f11192f4dc25" } new_table_name: "default.Table_12cc527a0517451f9a6f226194c4ca68" modify_external_catalogs: true
I20260925 20:00:31.608364 5354 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44610:
name: "default.Table_f9d13749640c4f11913f7449b48cf6c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:31.600374 5594 raft_consensus.cc:2243] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:31.609328 4784 tablet_service.cc:1558] Processing DeleteTablet for tablet f4b5baba575d4538b807a69eff0d90dd with delete_type TABLET_DATA_DELETED (replaced by tablet dbdd993621c4410386d7914283809577) from {username='slave'} at 127.0.0.1:53746
I20260925 20:00:31.609359 5594 raft_consensus.cc:2272] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:31.609566 5571 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:31.609761 5571 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:31.613103 4785 tablet_service.cc:1558] Processing DeleteTablet for tablet 983d375d3fda4e63ab93b6cde278e187 with delete_type TABLET_DATA_DELETED (replaced by tablet b321f36472cc42babf1ec3f64ab990ed) from {username='slave'} at 127.0.0.1:53746
I20260925 20:00:31.615921 5593 tablet_replica.cc:333] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:31.616173 5593 raft_consensus.cc:2243] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:31.616695 5593 raft_consensus.cc:2272] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:31.617344 5593 ts_tablet_manager.cc:1916] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:31.619208 4505 tablet_service.cc:1511] Processing CreateTablet for tablet dbdd993621c4410386d7914283809577 (DEFAULT_TABLE table=default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:31.619657 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbdd993621c4410386d7914283809577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:31.618534 4644 tablet_service.cc:1511] Processing CreateTablet for tablet 83c0e7bd6ef040b9af169d9677137836 (DEFAULT_TABLE table=default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:31.622046 4644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83c0e7bd6ef040b9af169d9677137836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:31.622164 4504 tablet_service.cc:1511] Processing CreateTablet for tablet b321f36472cc42babf1ec3f64ab990ed (DEFAULT_TABLE table=default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:31.622728 4504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b321f36472cc42babf1ec3f64ab990ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:31.620054 4647 tablet_service.cc:1511] Processing CreateTablet for tablet dbdd993621c4410386d7914283809577 (DEFAULT_TABLE table=default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:31.624792 4647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbdd993621c4410386d7914283809577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:31.621409 4646 tablet_service.cc:1511] Processing CreateTablet for tablet b321f36472cc42babf1ec3f64ab990ed (DEFAULT_TABLE table=default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:31.628791 4646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b321f36472cc42babf1ec3f64ab990ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:31.632917 5593 ts_tablet_manager.cc:1929] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:31.633064 5593 log.cc:1199] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/f4b5baba575d4538b807a69eff0d90dd
I20260925 20:00:31.633498 5593 ts_tablet_manager.cc:1950] T f4b5baba575d4538b807a69eff0d90dd P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:31.634624 5593 tablet_replica.cc:333] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:31.634842 5593 raft_consensus.cc:2243] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:31.634931 5343 catalog_manager.cc:5063] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): tablet f4b5baba575d4538b807a69eff0d90dd (table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]) successfully deleted
I20260925 20:00:31.635046 5593 raft_consensus.cc:2272] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:31.618760 4506 tablet_service.cc:1511] Processing CreateTablet for tablet 83c0e7bd6ef040b9af169d9677137836 (DEFAULT_TABLE table=default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:31.635628 5593 ts_tablet_manager.cc:1916] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:31.635728 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83c0e7bd6ef040b9af169d9677137836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:31.640722 4785 tablet_service.cc:1511] Processing CreateTablet for tablet 83c0e7bd6ef040b9af169d9677137836 (DEFAULT_TABLE table=default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:31.641165 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83c0e7bd6ef040b9af169d9677137836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:31.643069 5593 ts_tablet_manager.cc:1929] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:31.643214 5593 log.cc:1199] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/983d375d3fda4e63ab93b6cde278e187
I20260925 20:00:31.643596 5593 ts_tablet_manager.cc:1950] T 983d375d3fda4e63ab93b6cde278e187 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:31.613260 5594 ts_tablet_manager.cc:1916] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:31.644660 5343 catalog_manager.cc:5063] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): tablet 983d375d3fda4e63ab93b6cde278e187 (table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]) successfully deleted
I20260925 20:00:31.648471 5599 tablet_bootstrap.cc:492] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:31.648535 4782 tablet_service.cc:1511] Processing CreateTablet for tablet dbdd993621c4410386d7914283809577 (DEFAULT_TABLE table=default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:31.649006 4782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbdd993621c4410386d7914283809577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:31.649554 5598 tablet_bootstrap.cc:492] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:31.650911 5599 tablet_bootstrap.cc:654] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:31.651523 5598 tablet_bootstrap.cc:654] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:31.659308 4782 tablet_service.cc:1511] Processing CreateTablet for tablet b321f36472cc42babf1ec3f64ab990ed (DEFAULT_TABLE table=default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:31.659741 4782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b321f36472cc42babf1ec3f64ab990ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:31.665863 5596 tablet_replica.cc:333] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:31.666150 5596 raft_consensus.cc:2243] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:31.666483 5596 raft_consensus.cc:2272] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:31.667236 5596 ts_tablet_manager.cc:1916] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:31.668946 5601 tablet_bootstrap.cc:492] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:31.664597 5594 ts_tablet_manager.cc:1929] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:31.670449 5594 log.cc:1199] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/f4b5baba575d4538b807a69eff0d90dd
I20260925 20:00:31.671061 5594 ts_tablet_manager.cc:1950] T f4b5baba575d4538b807a69eff0d90dd P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:31.671572 5601 tablet_bootstrap.cc:654] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
@ 0x7f364590081c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260925 20:00:31.677699 5341 catalog_manager.cc:5063] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): tablet f4b5baba575d4538b807a69eff0d90dd (table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]) successfully deleted
I20260925 20:00:31.678138 5594 tablet_replica.cc:333] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:31.678454 5594 raft_consensus.cc:2243] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:31.678699 5594 raft_consensus.cc:2272] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:31.679497 5601 tablet_bootstrap.cc:492] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:31.679529 5594 ts_tablet_manager.cc:1916] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:31.679672 5601 ts_tablet_manager.cc:1403] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.005s
I20260925 20:00:31.680716 5601 raft_consensus.cc:359] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:31.680956 5601 raft_consensus.cc:385] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:31.681057 5601 raft_consensus.cc:740] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:31.681442 5601 consensus_queue.cc:260] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:31.681806 5598 tablet_bootstrap.cc:492] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:31.681968 5598 ts_tablet_manager.cc:1403] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.000s
I20260925 20:00:31.682986 5598 raft_consensus.cc:359] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
I20260925 20:00:31.683223 5598 raft_consensus.cc:385] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:31.683326 5598 raft_consensus.cc:740] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:31.683669 5598 consensus_queue.cc:260] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
I20260925 20:00:31.686884 5598 ts_tablet_manager.cc:1434] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260925 20:00:31.687197 5598 tablet_bootstrap.cc:492] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:31.689126 5601 ts_tablet_manager.cc:1434] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260925 20:00:31.689498 5601 tablet_bootstrap.cc:492] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:31.689949 5599 tablet_bootstrap.cc:492] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:31.690116 5599 ts_tablet_manager.cc:1403] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.042s user 0.011s sys 0.026s
I20260925 20:00:31.691053 5599 raft_consensus.cc:359] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
I20260925 20:00:31.691267 5599 raft_consensus.cc:385] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:31.691361 5599 raft_consensus.cc:740] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:31.691522 5601 tablet_bootstrap.cc:654] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:31.691648 5599 consensus_queue.cc:260] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
I20260925 20:00:31.692328 4852 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:31.694564 5598 tablet_bootstrap.cc:654] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:31.694643 5599 ts_tablet_manager.cc:1434] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260925 20:00:31.694957 5599 tablet_bootstrap.cc:492] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:31.696910 5599 tablet_bootstrap.cc:654] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:31.698348 5596 ts_tablet_manager.cc:1929] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:31.698499 5596 log.cc:1199] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/f4b5baba575d4538b807a69eff0d90dd
I20260925 20:00:31.699019 5596 ts_tablet_manager.cc:1950] T f4b5baba575d4538b807a69eff0d90dd P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:31.702903 5343 catalog_manager.cc:5063] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395): tablet f4b5baba575d4538b807a69eff0d90dd (table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]) successfully deleted
I20260925 20:00:31.704200 5596 tablet_replica.cc:333] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:31.704480 5596 raft_consensus.cc:2243] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:31.705583 5596 raft_consensus.cc:2272] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:31.705904 5599 tablet_bootstrap.cc:492] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:31.706099 5599 ts_tablet_manager.cc:1403] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260925 20:00:31.706183 5596 ts_tablet_manager.cc:1916] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:31.707103 5599 raft_consensus.cc:359] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:31.707433 5599 raft_consensus.cc:385] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:31.707537 5599 raft_consensus.cc:740] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:31.707958 5599 consensus_queue.cc:260] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:31.708572 5599 ts_tablet_manager.cc:1434] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260925 20:00:31.709065 5599 tablet_bootstrap.cc:492] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
@ 0x7f3645900699 std::_Function_handler<>::_M_invoke() at ??:0
I20260925 20:00:31.711380 5599 tablet_bootstrap.cc:654] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:31.712983 4572 heartbeater.cc:344] Connected to a master server at 127.3.199.254:41289
I20260925 20:00:31.715443 5511 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" instance_seqno: 1790366418849966) as {username='slave'} at 127.3.199.193:47411; Asking this server to re-register.
I20260925 20:00:31.716336 4572 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:31.717113 5601 tablet_bootstrap.cc:492] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:31.717218 5598 tablet_bootstrap.cc:492] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:31.717281 5601 ts_tablet_manager.cc:1403] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.028s user 0.004s sys 0.002s
I20260925 20:00:31.717415 5598 ts_tablet_manager.cc:1403] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.030s user 0.000s sys 0.003s
I20260925 20:00:31.718134 5511 ts_manager.cc:194] Registered new tserver with Master: 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:31.718075 5601 raft_consensus.cc:359] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:31.718266 5601 raft_consensus.cc:385] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:31.718189 5598 raft_consensus.cc:359] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:31.718341 5601 raft_consensus.cc:740] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:31.718405 5598 raft_consensus.cc:385] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:31.718498 5598 raft_consensus.cc:740] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:31.718657 5601 consensus_queue.cc:260] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:31.718766 5596 ts_tablet_manager.cc:1929] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:31.718948 5596 log.cc:1199] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/983d375d3fda4e63ab93b6cde278e187
I20260925 20:00:31.718832 5598 consensus_queue.cc:260] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:31.719580 5596 ts_tablet_manager.cc:1950] T 983d375d3fda4e63ab93b6cde278e187 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:31.719587 5601 ts_tablet_manager.cc:1434] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260925 20:00:31.719939 5601 tablet_bootstrap.cc:492] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:31.722245 5601 tablet_bootstrap.cc:654] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:31.726331 5598 ts_tablet_manager.cc:1434] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260925 20:00:31.727399 5343 catalog_manager.cc:5063] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395): tablet 983d375d3fda4e63ab93b6cde278e187 (table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]) successfully deleted
I20260925 20:00:31.731372 5598 tablet_bootstrap.cc:492] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:31.731561 5601 tablet_bootstrap.cc:492] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:31.731719 5601 ts_tablet_manager.cc:1403] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260925 20:00:31.732612 5601 raft_consensus.cc:359] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
I20260925 20:00:31.732820 5601 raft_consensus.cc:385] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:31.732928 5601 raft_consensus.cc:740] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:31.733263 5601 consensus_queue.cc:260] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
@ 0xaaef9d std::function<>::operator()() at ??:?
I20260925 20:00:31.737568 5601 ts_tablet_manager.cc:1434] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260925 20:00:31.740414 5598 tablet_bootstrap.cc:654] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:31.742699 5594 ts_tablet_manager.cc:1929] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:31.742849 5594 log.cc:1199] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/983d375d3fda4e63ab93b6cde278e187
I20260925 20:00:31.743921 5599 tablet_bootstrap.cc:492] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:31.744022 5594 ts_tablet_manager.cc:1950] T 983d375d3fda4e63ab93b6cde278e187 P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:31.744081 5599 ts_tablet_manager.cc:1403] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.035s user 0.018s sys 0.007s
I20260925 20:00:31.744902 5599 raft_consensus.cc:359] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:31.745098 5599 raft_consensus.cc:385] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:31.745194 5599 raft_consensus.cc:740] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:31.745299 5341 catalog_manager.cc:5063] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): tablet 983d375d3fda4e63ab93b6cde278e187 (table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]) successfully deleted
I20260925 20:00:31.745496 5599 consensus_queue.cc:260] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:31.746080 5599 ts_tablet_manager.cc:1434] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
@ 0x7f363194b079 kudu::Thread::SuperviseThread() at ??:0
I20260925 20:00:31.747082 5598 tablet_bootstrap.cc:492] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:31.747220 5598 ts_tablet_manager.cc:1403] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.002s
I20260925 20:00:31.748047 5598 raft_consensus.cc:359] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:31.748231 5598 raft_consensus.cc:385] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:31.748358 5598 raft_consensus.cc:740] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:31.748592 5598 consensus_queue.cc:260] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:31.749174 5598 ts_tablet_manager.cc:1434] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
@ 0x7f36350216db start_thread at ??:0
@ 0x7f362daa861f clone at ??:0
I20260925 20:00:31.868013 5575 raft_consensus.cc:493] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:31.868216 5575 raft_consensus.cc:515] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
I20260925 20:00:31.869079 5575 leader_election.cc:290] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:31.869903 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83c0e7bd6ef040b9af169d9677137836" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:31.870147 4526 raft_consensus.cc:2468] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:31.870726 4665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83c0e7bd6ef040b9af169d9677137836" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:31.870993 4665 raft_consensus.cc:2468] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:31.871961 4741 leader_election.cc:304] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:31.872480 5478 raft_consensus.cc:493] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:31.872629 5478 raft_consensus.cc:515] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:31.873417 5478 leader_election.cc:290] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:31.873586 5325 raft_consensus.cc:2804] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:31.873716 5325 raft_consensus.cc:493] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:31.873829 5325 raft_consensus.cc:3060] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:31.874495 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdd993621c4410386d7914283809577" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" is_pre_election: true
I20260925 20:00:31.874733 4805 raft_consensus.cc:2468] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 0.
I20260925 20:00:31.875200 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdd993621c4410386d7914283809577" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:31.875408 4526 raft_consensus.cc:2468] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 0.
I20260925 20:00:31.875963 4603 leader_election.cc:304] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:31.876335 5478 raft_consensus.cc:2804] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:31.876449 5478 raft_consensus.cc:493] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:31.876528 5478 raft_consensus.cc:3060] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:31.877275 5325 raft_consensus.cc:515] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
I20260925 20:00:31.878010 5325 leader_election.cc:290] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Requested vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:31.879596 5478 raft_consensus.cc:515] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:31.880748 4665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83c0e7bd6ef040b9af169d9677137836" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:31.880976 4665 raft_consensus.cc:3060] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:31.885392 4526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83c0e7bd6ef040b9af169d9677137836" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:31.885679 4526 raft_consensus.cc:3060] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:31.889040 4526 raft_consensus.cc:2468] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:31.889235 4665 raft_consensus.cc:2468] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:31.889802 4741 leader_election.cc:304] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:31.890041 5478 leader_election.cc:290] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 election: Requested vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:31.890151 5325 raft_consensus.cc:2804] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:31.890357 5325 raft_consensus.cc:697] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Becoming Leader. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Running, Role: LEADER
I20260925 20:00:31.890673 5325 consensus_queue.cc:237] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } }
I20260925 20:00:31.891233 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdd993621c4410386d7914283809577" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
I20260925 20:00:31.891419 4805 raft_consensus.cc:3060] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:31.894712 4805 raft_consensus.cc:2468] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 1.
I20260925 20:00:31.895978 4603 leader_election.cc:304] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:31.896889 5478 raft_consensus.cc:2804] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:31.896935 4524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdd993621c4410386d7914283809577" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:31.897289 4524 raft_consensus.cc:3060] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:31.900991 4524 raft_consensus.cc:2468] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 1.
I20260925 20:00:31.912592 5478 raft_consensus.cc:697] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 LEADER]: Becoming Leader. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Running, Role: LEADER
I20260925 20:00:31.913048 5478 consensus_queue.cc:237] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
W20260925 20:00:31.972239 4862 connection.cc:573] client connection to 127.3.199.253:41897 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:31.973521 4862 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.3.199.253:41897)
W20260925 20:00:31.974406 4862 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.199.253:41897)
W20260925 20:00:31.974952 4862 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.3.199.253:41897)
W20260925 20:00:31.975756 4862 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.199.253:41897)
I20260925 20:00:32.115597 5625 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:32.115787 5625 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:32.117702 5625 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:32.118539 5497 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:32.120155 5497 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:32.120733 5498 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:32.120941 5498 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:32.121388 5625 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260925 20:00:32.134286 5469 raft_consensus.cc:493] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:32.134573 5469 raft_consensus.cc:515] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:32.135551 5469 leader_election.cc:290] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:32.136404 4665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b321f36472cc42babf1ec3f64ab990ed" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:32.136495 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b321f36472cc42babf1ec3f64ab990ed" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" is_pre_election: true
I20260925 20:00:32.136646 4665 raft_consensus.cc:2468] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:32.136822 4805 raft_consensus.cc:2468] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 0.
I20260925 20:00:32.137176 4455 leader_election.cc:304] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:32.137487 5469 raft_consensus.cc:2804] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:32.137604 5469 raft_consensus.cc:493] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:32.137668 5469 raft_consensus.cc:3060] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:32.139719 5469 raft_consensus.cc:515] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:32.140492 5469 leader_election.cc:290] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Requested vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:32.141032 4805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b321f36472cc42babf1ec3f64ab990ed" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2623c3b2c3497ca1df4a676fa6807e"
I20260925 20:00:32.141283 4665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b321f36472cc42babf1ec3f64ab990ed" candidate_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:32.141316 4805 raft_consensus.cc:3060] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:32.141513 4665 raft_consensus.cc:3060] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:32.143858 4805 raft_consensus.cc:2468] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
I20260925 20:00:32.143879 4665 raft_consensus.cc:2468] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145cdcf7953e4c0d9686b1c6a86f9482 in term 1.
I20260925 20:00:32.144402 4455 leader_election.cc:304] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
I20260925 20:00:32.144757 5469 raft_consensus.cc:2804] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:32.145159 5469 raft_consensus.cc:697] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 LEADER]: Becoming Leader. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Running, Role: LEADER
I20260925 20:00:32.145439 5469 consensus_queue.cc:237] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:32.634014 5631 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:32.634212 5631 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:32.634884 5631 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:32.637277 5498 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:32.637727 5497 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:32.637895 5497 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:32.638163 5631 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260925 20:00:32.707330 5478 consensus_queue.cc:1048] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:32.715583 5638 consensus_queue.cc:1048] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:32.765466 5325 consensus_queue.cc:1048] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:32.771785 5645 consensus_queue.cc:1048] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:33.066473 5649 consensus_queue.cc:1048] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:33.066851 5469 consensus_queue.cc:1048] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:33.572966 5654 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:33.573237 5654 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:33.574088 5654 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:33.576327 5497 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:33.576615 5498 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:33.576787 5498 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:33.577095 5654 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260925 20:00:33.614071 3871 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu pid 5477
I20260925 20:00:33.622085 5541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 20:00:33.742411 3871 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu with pid 5477
I20260925 20:00:34.347858 3871 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
/tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.254:41289
--webserver_interface=127.3.199.254
--webserver_port=32859
--builtin_ntp_servers=127.3.199.212:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260925 20:00:34.735370 5676 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 20:00:34.735769 5676 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 20:00:34.735898 5676 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 20:00:34.745493 5676 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 20:00:34.745630 5676 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 20:00:34.745733 5676 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 20:00:34.745828 5676 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 20:00:34.756891 5676 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.199.212:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.254:41289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.199.254
--webserver_port=32859
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 25 Sep 2026 19:43:21 UTC on bdcb31816ec0
build id 13997
ASAN enabled
I20260925 20:00:34.758754 5676 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 20:00:34.760696 5676 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 20:00:34.769761 5689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:34.769731 5688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:34.770354 5691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 20:00:34.771066 5676 server_base.cc:1061] running on GCE node
I20260925 20:00:34.771939 5676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 20:00:34.773502 5676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 20:00:34.774799 5676 hybrid_clock.cc:648] HybridClock initialized: now 1790366434774731 us; error 56 us; skew 500 ppm
I20260925 20:00:34.775213 5676 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 20:00:34.777695 5676 webserver.cc:533] Webserver started at http://127.3.199.254:32859/ using document root <none> and password file <none>
I20260925 20:00:34.778661 5676 fs_manager.cc:362] Metadata directory not provided
I20260925 20:00:34.778800 5676 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 20:00:34.785225 5676 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 20:00:34.788549 5697 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:34.790272 5676 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260925 20:00:34.790444 5676 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data,/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
uuid: "169b6de20560440b99cc08af549ce477"
format_stamp: "Formatted at 2026-09-25 20:00:16 on dist-test-slave-9gwk"
I20260925 20:00:34.791215 5676 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 20:00:34.810391 5676 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 20:00:34.811453 5676 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 20:00:34.811728 5676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 20:00:34.834401 5748 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.199.254:41289 every 8 connection(s)
I20260925 20:00:34.834405 5676 rpc_server.cc:307] RPC server started. Bound to: 127.3.199.254:41289
I20260925 20:00:34.836581 5676 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data/info.pb
I20260925 20:00:34.840474 3871 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu as pid 5676
I20260925 20:00:34.845266 5749 sys_catalog.cc:263] Verifying existing consensus state
I20260925 20:00:34.849323 5749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Bootstrap starting.
I20260925 20:00:34.871387 5749 log.cc:826] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Log is configured to *not* fsync() on all Append() calls
I20260925 20:00:34.946511 5749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Bootstrap replayed 1/1 log segments. Stats: ops{read=91 overwritten=0 applied=91 ignored=0} inserts{seen=55 ignored=0} mutations{seen=68 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 20:00:34.947148 5749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Bootstrap complete.
I20260925 20:00:34.952857 5749 raft_consensus.cc:359] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:34.953606 5749 raft_consensus.cc:740] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 169b6de20560440b99cc08af549ce477, State: Initialized, Role: FOLLOWER
I20260925 20:00:34.954422 5749 consensus_queue.cc:260] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 91, Last appended: 7.91, Last appended by leader: 91, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:34.955295 5753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:34.955555 5753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:34.956434 5749 sys_catalog.cc:565] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 20:00:34.975701 5767 catalog_manager.cc:1308] Loaded cluster ID: 7efb83138ca94380bf2d8f0f2f91c381
I20260925 20:00:34.975940 5767 catalog_manager.cc:1601] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: loading cluster ID for follower catalog manager: success
I20260925 20:00:34.981717 5767 catalog_manager.cc:1623] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: acquiring CA information for follower catalog manager: success
I20260925 20:00:34.985666 5767 catalog_manager.cc:1651] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 20:00:34.986542 5676 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:00:34.992481 5676 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:00:35.112829 4572 heartbeater.cc:344] Connected to a master server at 127.3.199.254:41289
I20260925 20:00:35.116745 5714 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" instance_seqno: 1790366418849966) as {username='slave'} at 127.3.199.193:49129; Asking this server to re-register.
I20260925 20:00:35.117717 4572 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:35.119899 5714 ts_manager.cc:194] Registered new tserver with Master: 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:35.196508 4713 heartbeater.cc:344] Connected to a master server at 127.3.199.254:41289
I20260925 20:00:35.198565 5714 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" instance_seqno: 1790366419525613) as {username='slave'} at 127.3.199.194:56073; Asking this server to re-register.
I20260925 20:00:35.199254 4713 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:35.200836 5714 ts_manager.cc:194] Registered new tserver with Master: 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:35.254576 4851 heartbeater.cc:344] Connected to a master server at 127.3.199.254:41289
I20260925 20:00:35.256322 5714 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" instance_seqno: 1790366420065491) as {username='slave'} at 127.3.199.195:46169; Asking this server to re-register.
I20260925 20:00:35.256990 4851 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:35.259208 5714 ts_manager.cc:194] Registered new tserver with Master: dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:35.439656 5753 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:35.440073 5753 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:35.442785 5753 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:35.443441 5700 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:35.444949 5701 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:35.445163 5700 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:35.445320 5700 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:35.445717 5753 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260925 20:00:36.068938 5777 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:36.069128 5777 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:36.069906 5777 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:36.072793 5700 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:36.073020 5701 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:36.073189 5701 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:36.073455 5777 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260925 20:00:36.725827 5790 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:36.726101 5790 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:36.726918 5790 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:36.729542 5701 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:36.729795 5700 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:36.729938 5700 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:36.730198 5790 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260925 20:00:37.640236 5798 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:37.640530 5798 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:37.641184 5798 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:37.643477 5700 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:37.643677 5701 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:37.643793 5701 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:37.644068 5798 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260925 20:00:38.092947 3871 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
/tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.253:41897
--webserver_interface=127.3.199.253
--webserver_port=41541
--builtin_ntp_servers=127.3.199.212:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260925 20:00:38.500885 5807 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 20:00:38.501339 5807 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 20:00:38.501497 5807 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 20:00:38.510712 5807 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 20:00:38.510895 5807 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 20:00:38.511003 5807 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 20:00:38.511087 5807 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 20:00:38.524019 5807 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.199.212:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.253:41897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.199.253
--webserver_port=41541
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 25 Sep 2026 19:43:21 UTC on bdcb31816ec0
build id 13997
ASAN enabled
I20260925 20:00:38.525959 5807 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 20:00:38.528366 5807 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 20:00:38.539146 5814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:38.539585 5817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:38.539306 5815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 20:00:38.541332 5807 server_base.cc:1061] running on GCE node
I20260925 20:00:38.542385 5807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 20:00:38.544265 5807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 20:00:38.545608 5807 hybrid_clock.cc:648] HybridClock initialized: now 1790366438545552 us; error 47 us; skew 500 ppm
I20260925 20:00:38.546128 5807 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 20:00:38.548880 5807 webserver.cc:533] Webserver started at http://127.3.199.253:41541/ using document root <none> and password file <none>
I20260925 20:00:38.549742 5807 fs_manager.cc:362] Metadata directory not provided
I20260925 20:00:38.549896 5807 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 20:00:38.557308 5807 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 20:00:38.561724 5823 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:38.563966 5807 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260925 20:00:38.564142 5807 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data,/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
uuid: "da29ee550f82456fa7c5ce7ea916909f"
format_stamp: "Formatted at 2026-09-25 20:00:17 on dist-test-slave-9gwk"
I20260925 20:00:38.565301 5807 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 20:00:38.604030 5807 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 20:00:38.605278 5807 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 20:00:38.605659 5807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 20:00:38.633013 5807 rpc_server.cc:307] RPC server started. Bound to: 127.3.199.253:41897
I20260925 20:00:38.633193 5874 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.199.253:41897 every 8 connection(s)
I20260925 20:00:38.635242 5807 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
I20260925 20:00:38.636729 3871 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu as pid 5807
I20260925 20:00:38.644802 5875 sys_catalog.cc:263] Verifying existing consensus state
I20260925 20:00:38.649413 5875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Bootstrap starting.
I20260925 20:00:38.669008 5875 log.cc:826] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Log is configured to *not* fsync() on all Append() calls
I20260925 20:00:38.721953 4852 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:38.722193 4852 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:38.723119 4714 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:38.758451 5875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Bootstrap replayed 1/1 log segments. Stats: ops{read=91 overwritten=0 applied=91 ignored=0} inserts{seen=55 ignored=0} mutations{seen=68 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 20:00:38.759301 5875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Bootstrap complete.
I20260925 20:00:38.766710 5875 raft_consensus.cc:359] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:38.767494 5875 raft_consensus.cc:740] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: da29ee550f82456fa7c5ce7ea916909f, State: Initialized, Role: FOLLOWER
I20260925 20:00:38.768234 5875 consensus_queue.cc:260] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 91, Last appended: 7.91, Last appended by leader: 91, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:38.769198 5883 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:38.769549 5883 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:38.770529 5875 sys_catalog.cc:565] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 20:00:38.777863 5897 catalog_manager.cc:818] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 20:00:38.803174 5807 master_runner.cc:187] Error getting master registration for 127.3.199.252:37957: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111),
I20260925 20:00:38.806797 5807 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:00:39.122067 4573 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:39.124541 5840 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" instance_seqno: 1790366418849966) as {username='slave'} at 127.3.199.193:39687; Asking this server to re-register.
I20260925 20:00:39.125416 4573 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:39.127435 5840 ts_manager.cc:194] Registered new tserver with Master: 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:39.163599 5883 raft_consensus.cc:493] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:39.163913 5883 raft_consensus.cc:515] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:39.166040 5883 leader_election.cc:290] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:39.166648 5724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da29ee550f82456fa7c5ce7ea916909f" candidate_term: 8 candidate_status { last_received { term: 7 index: 91 } } ignore_live_leader: false dest_uuid: "169b6de20560440b99cc08af549ce477" is_pre_election: true
I20260925 20:00:39.167070 5724 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da29ee550f82456fa7c5ce7ea916909f in term 7.
W20260925 20:00:39.167040 5827 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:39.168063 5826 leader_election.cc:304] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 169b6de20560440b99cc08af549ce477, da29ee550f82456fa7c5ce7ea916909f; no voters:
I20260925 20:00:39.168521 5883 raft_consensus.cc:2804] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260925 20:00:39.168684 5883 raft_consensus.cc:493] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:39.168803 5883 raft_consensus.cc:3060] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 7 FOLLOWER]: Advancing to term 8
W20260925 20:00:39.168851 5827 leader_election.cc:336] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:39.172983 5883 raft_consensus.cc:515] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:39.173741 5883 leader_election.cc:290] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 8 election: Requested vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:39.174404 5724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da29ee550f82456fa7c5ce7ea916909f" candidate_term: 8 candidate_status { last_received { term: 7 index: 91 } } ignore_live_leader: false dest_uuid: "169b6de20560440b99cc08af549ce477"
I20260925 20:00:39.174664 5724 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 7 FOLLOWER]: Advancing to term 8
W20260925 20:00:39.175464 5827 leader_election.cc:336] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:39.178716 5724 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da29ee550f82456fa7c5ce7ea916909f in term 8.
I20260925 20:00:39.179272 5826 leader_election.cc:304] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 169b6de20560440b99cc08af549ce477, da29ee550f82456fa7c5ce7ea916909f; no voters: 7f43a6a14ef547f2aeb22ce2b334383e
I20260925 20:00:39.179667 5883 raft_consensus.cc:2804] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 8 FOLLOWER]: Leader election won for term 8
I20260925 20:00:39.180097 5883 raft_consensus.cc:697] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 8 LEADER]: Becoming Leader. State: Replica: da29ee550f82456fa7c5ce7ea916909f, State: Running, Role: LEADER
I20260925 20:00:39.180685 5883 consensus_queue.cc:237] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 91, Committed index: 91, Last appended: 7.91, Last appended by leader: 91, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:39.183534 5904 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: New leader da29ee550f82456fa7c5ce7ea916909f. Latest consensus state: current_term: 8 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:39.183869 5904 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:39.185129 5906 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260925 20:00:39.190341 5906 catalog_manager.cc:692] Loaded metadata for table default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330]
I20260925 20:00:39.190833 5906 catalog_manager.cc:692] Loaded metadata for table default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf]
I20260925 20:00:39.191247 5906 catalog_manager.cc:692] Loaded metadata for table default.table_703616ef44ba43f9afb9c4df73eb372c [id=25f6e1ade43c4519bed519037ee8f761]
I20260925 20:00:39.191603 5906 catalog_manager.cc:692] Loaded metadata for table default.table_667eace0e712438bbd1211f7e3e665f8 [id=439b3d65790141d1abfbca4788836f80]
I20260925 20:00:39.191956 5906 catalog_manager.cc:692] Loaded metadata for table default.table_127c3d62fd1c4394bcd55bc6fc18bc5d [id=45cbddfbaced49a989c1fb61a644a713]
I20260925 20:00:39.192366 5906 catalog_manager.cc:692] Loaded metadata for table default.table_2ebd7ecb76b44d13a6847408f7fd7d17 [id=4b80e3afa6954e2ea8b6d1b5ec79a9d1]
I20260925 20:00:39.192715 5906 catalog_manager.cc:692] Loaded metadata for table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]
I20260925 20:00:39.193065 5906 catalog_manager.cc:692] Loaded metadata for table default.table_c78f9cc1d7e74b51b29858c7d3248fb8 [id=7ead82097c49448f85b1dbef027be066]
I20260925 20:00:39.193403 5906 catalog_manager.cc:692] Loaded metadata for table default.table_098b200909f842c88a38f11192f4dc25 [id=86f1c8a16cc14ef9b72c95060313b3ad]
I20260925 20:00:39.193823 5906 catalog_manager.cc:692] Loaded metadata for table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]
I20260925 20:00:39.194223 5906 catalog_manager.cc:692] Loaded metadata for table default.table_5b8386185e8e4fcd99110bd5137a578c [id=a385e0b7db6a49e8b1baf34a5fdd3473]
I20260925 20:00:39.194579 5906 catalog_manager.cc:692] Loaded metadata for table default.table_7722a249d10f4616ba5b5bb2fc149c18 [id=a8f568d7faab469ba7ea8a2a38bfd638]
I20260925 20:00:39.194964 5906 catalog_manager.cc:692] Loaded metadata for table default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d]
I20260925 20:00:39.195364 5906 catalog_manager.cc:692] Loaded metadata for table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]
I20260925 20:00:39.195868 5906 catalog_manager.cc:692] Loaded metadata for table default.table_08807903f9884f2a9707d4066e7f2f52 [id=f6fa165ad5e7471589cb63403548ab2c]
I20260925 20:00:39.198280 5906 tablet_loader.cc:96] loaded metadata for tablet 0827e2c7eff041369a94c31abdf9b093 (table default.table_127c3d62fd1c4394bcd55bc6fc18bc5d [id=45cbddfbaced49a989c1fb61a644a713])
I20260925 20:00:39.199023 5906 tablet_loader.cc:96] loaded metadata for tablet 2ddb9fbd0aeb4c4a985508ac67c93227 (table default.table_7722a249d10f4616ba5b5bb2fc149c18 [id=a8f568d7faab469ba7ea8a2a38bfd638])
I20260925 20:00:39.199383 5906 tablet_loader.cc:96] loaded metadata for tablet 44f42691903d4e2bb3f24ecb75a3f86e (table default.table_08807903f9884f2a9707d4066e7f2f52 [id=f6fa165ad5e7471589cb63403548ab2c])
I20260925 20:00:39.199842 5906 tablet_loader.cc:96] loaded metadata for tablet 5982cfa34e4e40d3894cbf46aab33715 (table default.table_2ebd7ecb76b44d13a6847408f7fd7d17 [id=4b80e3afa6954e2ea8b6d1b5ec79a9d1])
I20260925 20:00:39.200189 5906 tablet_loader.cc:96] loaded metadata for tablet 59a14370409146a1847c5b76755e3feb (table default.table_c78f9cc1d7e74b51b29858c7d3248fb8 [id=7ead82097c49448f85b1dbef027be066])
I20260925 20:00:39.200868 5906 tablet_loader.cc:96] loaded metadata for tablet 653a4121e8934471b99a9853b73617ab (table default.table_5b8386185e8e4fcd99110bd5137a578c [id=a385e0b7db6a49e8b1baf34a5fdd3473])
I20260925 20:00:39.201174 5906 tablet_loader.cc:96] loaded metadata for tablet 72c3c64108f348a5ab893e6954d72d77 (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812])
I20260925 20:00:39.201488 5906 tablet_loader.cc:96] loaded metadata for tablet 83c0e7bd6ef040b9af169d9677137836 (table default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330])
I20260925 20:00:39.201952 5906 tablet_loader.cc:96] loaded metadata for tablet 8d0bc4cceef042389ad99a3515aa0f43 (table default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d])
I20260925 20:00:39.202595 5906 tablet_loader.cc:96] loaded metadata for tablet b321f36472cc42babf1ec3f64ab990ed (table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7])
I20260925 20:00:39.202930 5906 tablet_loader.cc:96] loaded metadata for tablet b59f74f459c542a88e8bef5f1c51357d (table default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf])
I20260925 20:00:39.203239 5906 tablet_loader.cc:96] loaded metadata for tablet b65400989b944ca988d6f26838277336 (table default.table_667eace0e712438bbd1211f7e3e665f8 [id=439b3d65790141d1abfbca4788836f80])
I20260925 20:00:39.203734 5906 tablet_loader.cc:96] loaded metadata for tablet dbdd993621c4410386d7914283809577 (table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22])
I20260925 20:00:39.204344 5906 tablet_loader.cc:96] loaded metadata for tablet e0648fcd521e4db19359c743447246b7 (table default.table_703616ef44ba43f9afb9c4df73eb372c [id=25f6e1ade43c4519bed519037ee8f761])
I20260925 20:00:39.204715 5906 tablet_loader.cc:96] loaded metadata for tablet ee843d55757347ef9942dffd917c0ff8 (table default.table_098b200909f842c88a38f11192f4dc25 [id=86f1c8a16cc14ef9b72c95060313b3ad])
I20260925 20:00:39.205360 5906 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260925 20:00:39.206817 5906 catalog_manager.cc:1308] Loaded cluster ID: 7efb83138ca94380bf2d8f0f2f91c381
I20260925 20:00:39.206941 5906 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260925 20:00:39.210871 5906 catalog_manager.cc:1553] Loading token signing keys...
I20260925 20:00:39.212560 5906 catalog_manager.cc:6230] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Loaded TSK: 0
I20260925 20:00:39.214224 5906 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260925 20:00:39.216513 5906 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260925 20:00:39.218372 5906 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: 20
I20260925 20:00:39.226934 4852 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:39.229135 5840 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" instance_seqno: 1790366419525613) as {username='slave'} at 127.3.199.194:56633; Asking this server to re-register.
I20260925 20:00:39.229656 5839 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" instance_seqno: 1790366420065491) as {username='slave'} at 127.3.199.195:55605; Asking this server to re-register.
I20260925 20:00:39.229971 4714 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:39.230255 4714 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:39.230450 4852 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:39.230690 4852 heartbeater.cc:507] Master 127.3.199.253:41897 requested a full tablet report, sending...
I20260925 20:00:39.232921 5839 ts_manager.cc:194] Registered new tserver with Master: dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:39.233108 5840 ts_manager.cc:194] Registered new tserver with Master: 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:39.235529 5839 catalog_manager.cc:5838] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e reported cstate change: term changed from 0 to 1, leader changed from <none> to 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193). New cstate: current_term: 1 leader_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } }
I20260925 20:00:39.236351 5839 catalog_manager.cc:5838] T 653a4121e8934471b99a9853b73617ab P dd2623c3b2c3497ca1df4a676fa6807e reported cstate change: term changed from 0 to 1, leader changed from <none> to 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193). New cstate: current_term: 1 leader_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } }
I20260925 20:00:39.236713 5839 catalog_manager.cc:5838] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e reported cstate change: term changed from 0 to 1, leader changed from <none> to 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193). New cstate: current_term: 1 leader_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } }
I20260925 20:00:39.237093 5839 catalog_manager.cc:5838] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e reported cstate change: term changed from 0 to 1, leader changed from <none> to dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195). New cstate: current_term: 1 leader_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } health_report { overall_health: HEALTHY } } }
I20260925 20:00:39.237757 5839 catalog_manager.cc:5838] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e reported cstate change: term changed from 0 to 1, leader changed from <none> to 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193). New cstate: current_term: 1 leader_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } }
I20260925 20:00:39.238034 5839 catalog_manager.cc:5938] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395) does not have the latest schema for tablet b321f36472cc42babf1ec3f64ab990ed (table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]). Expected version 1 got 0
I20260925 20:00:39.238440 5839 catalog_manager.cc:5838] T 5982cfa34e4e40d3894cbf46aab33715 P dd2623c3b2c3497ca1df4a676fa6807e reported cstate change: term changed from 0 to 1, leader changed from <none> to 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193). New cstate: current_term: 1 leader_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } }
I20260925 20:00:39.239007 5839 catalog_manager.cc:5838] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e reported cstate change: term changed from 0 to 1, leader changed from <none> to 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194). New cstate: current_term: 1 leader_uuid: "46a6f571dfa749b7905d8458d4c929b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } }
I20260925 20:00:39.239316 5839 catalog_manager.cc:5938] TS dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395) does not have the latest schema for tablet dbdd993621c4410386d7914283809577 (table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]). Expected version 1 got 0
I20260925 20:00:39.246485 5724 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Refusing update from remote peer da29ee550f82456fa7c5ce7ea916909f: Log matching property violated. Preceding OpId in replica: term: 7 index: 91. Preceding OpId from leader: term: 8 index: 93. (index mismatch)
W20260925 20:00:39.246960 5827 consensus_peers.cc:597] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f -> Peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Couldn't send request to peer 7f43a6a14ef547f2aeb22ce2b334383e. Status: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 20:00:39.247443 5904 consensus_queue.cc:1048] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [LEADER]: Connected to new peer: Peer: permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 92, Last known committed idx: 91, Time since last communication: 0.000s
I20260925 20:00:39.258821 5909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da29ee550f82456fa7c5ce7ea916909f. Latest consensus state: current_term: 8 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:39.259222 5909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:39.261360 5883 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 8 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:39.261674 5883 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:39.262722 5910 mvcc.cc:204] Tried to move back new op lower bound from 7333340935144316928 to 7333340934888001536. Current Snapshot: MvccSnapshot[applied={T|T < 7333340935144316928}]
I20260925 20:00:39.263882 5909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 8 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:39.264096 5909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:39.265965 5840 catalog_manager.cc:5938] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647) does not have the latest schema for tablet b321f36472cc42babf1ec3f64ab990ed (table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]). Expected version 1 got 0
I20260925 20:00:39.266633 5840 catalog_manager.cc:5938] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647) does not have the latest schema for tablet dbdd993621c4410386d7914283809577 (table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]). Expected version 1 got 0
I20260925 20:00:39.297281 5840 master_service.cc:960] ReplaceTablet: received request to replace tablet 8d0bc4cceef042389ad99a3515aa0f43 from {username='slave'} at 127.0.0.1:58570
I20260925 20:00:39.297981 5839 master_service.cc:960] ReplaceTablet: received request to replace tablet dbdd993621c4410386d7914283809577 from {username='slave'} at 127.0.0.1:58570
I20260925 20:00:39.309952 5839 catalog_manager.cc:6133] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Sending DeleteTablet for 3 replicas of tablet dbdd993621c4410386d7914283809577
I20260925 20:00:39.311396 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet dbdd993621c4410386d7914283809577 with delete_type TABLET_DATA_DELETED (replaced by tablet 81d09be24f0a4687a1b17ddd02fd4e47) from {username='slave'} at 127.0.0.1:45194
I20260925 20:00:39.312669 4644 tablet_service.cc:1558] Processing DeleteTablet for tablet dbdd993621c4410386d7914283809577 with delete_type TABLET_DATA_DELETED (replaced by tablet 81d09be24f0a4687a1b17ddd02fd4e47) from {username='slave'} at 127.0.0.1:35390
I20260925 20:00:39.317662 5838 master_service.cc:960] ReplaceTablet: received request to replace tablet 0827e2c7eff041369a94c31abdf9b093 from {username='slave'} at 127.0.0.1:58570
I20260925 20:00:39.320024 5831 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58570:
table { table_name: "default.Table_127c3d62fd1c4394bcd55bc6fc18bc5d" } modify_external_catalogs: true
I20260925 20:00:39.320430 5831 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58570:
table { table_name: "default.Table_127c3d62fd1c4394bcd55bc6fc18bc5d" } modify_external_catalogs: true
I20260925 20:00:39.320899 5837 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58570:
name: "default.Table_30a6c0d501b04bfc882cec2f43c06f83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:39.322086 5836 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58570:
name: "default.Table_9f19364a66794406806691ab3a701b72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:39.322742 5835 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58570:
table { table_name: "default.Table_2ebd7ecb76b44d13a6847408f7fd7d17" } modify_external_catalogs: true
I20260925 20:00:39.323056 5835 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58570:
table { table_name: "default.Table_2ebd7ecb76b44d13a6847408f7fd7d17" } modify_external_catalogs: true
I20260925 20:00:39.323477 5834 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58570:
table { table_name: "default.Table_667eace0e712438bbd1211f7e3e665f8" } modify_external_catalogs: true
I20260925 20:00:39.323519 5924 tablet.cc:1751] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 20:00:39.323670 5834 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58570:
table { table_name: "default.Table_667eace0e712438bbd1211f7e3e665f8" } modify_external_catalogs: true
I20260925 20:00:39.324085 5833 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58570:
name: "default.Table_aae983110c484d68a3a085faa1688c5e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:39.324831 5832 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58570:
name: "default.Table_4627e927cd124d049b0fdc22eb15c554"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:39.330046 5923 tablet.cc:1751] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 20:00:39.336840 5926 tablet_replica.cc:333] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:39.337090 5926 raft_consensus.cc:2243] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:39.340082 5925 tablet_replica.cc:333] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:39.364123 5926 pending_rounds.cc:70] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0: Trying to abort 1 pending ops.
I20260925 20:00:39.364246 5926 pending_rounds.cc:77] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7333340935352795136 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "dbdd993621c4410386d7914283809577" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.table_b74d63554f344195800ce065c2a3cddf" dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" new_extra_config { } }
W20260925 20:00:39.364800 5926 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35390: Aborted: Op aborted
I20260925 20:00:39.365373 5926 raft_consensus.cc:2272] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:39.366503 5840 catalog_manager.cc:6133] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Sending DeleteTablet for 3 replicas of tablet 8d0bc4cceef042389ad99a3515aa0f43
I20260925 20:00:39.367564 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d0bc4cceef042389ad99a3515aa0f43 with delete_type TABLET_DATA_DELETED (replaced by tablet 465db2a7c1fd48deb8a09d3366cfc04b) from {username='slave'} at 127.0.0.1:45194
I20260925 20:00:39.367827 5926 ts_tablet_manager.cc:1916] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:39.377287 5839 catalog_manager.cc:6934] ReplaceTablet: tablet dbdd993621c4410386d7914283809577 deleted and replaced by tablet 81d09be24f0a4687a1b17ddd02fd4e47
I20260925 20:00:39.378705 5926 ts_tablet_manager.cc:1929] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:39.378953 5926 log.cc:1199] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/dbdd993621c4410386d7914283809577
I20260925 20:00:39.379400 5926 ts_tablet_manager.cc:1950] T dbdd993621c4410386d7914283809577 P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:39.380632 5840 catalog_manager.cc:6934] ReplaceTablet: tablet 8d0bc4cceef042389ad99a3515aa0f43 deleted and replaced by tablet 465db2a7c1fd48deb8a09d3366cfc04b
I20260925 20:00:39.382344 5925 raft_consensus.cc:2243] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:39.383154 5925 pending_rounds.cc:70] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482: Trying to abort 1 pending ops.
I20260925 20:00:39.383229 5925 pending_rounds.cc:77] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7333340935352795136 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "dbdd993621c4410386d7914283809577" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.table_b74d63554f344195800ce065c2a3cddf" dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" new_extra_config { } }
I20260925 20:00:39.383860 5925 raft_consensus.cc:2272] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:39.391150 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 81d09be24f0a4687a1b17ddd02fd4e47 (DEFAULT_TABLE table=default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 20:00:39.390846 5931 consensus_queue.cc:1175] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" status { last_received { term: 1 index: 2 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 2 } }
F20260925 20:00:39.391661 5896 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 21, previous 21; dumping first 2 out of 13 received events: NotificationEvent(eventId=21, eventTime=1790366425, eventType=DROP_TABLE, dbName=default, tableName=table_30a6c0d501b04bfc882cec2f43c06f83, message={"server":"","servicePrincipal":"","db":"default","table":"table_30a6c0d501b04bfc882cec2f43c06f83","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_30a6c0d501b04bfc882cec2f43c06f83\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790366424},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/warehouse/table_30a6c0d501b04bfc882cec2f43c06f83\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_30a6c0d501b04bfc882cec2f43c06f83\",\"kudu.table_id\":\"44287e882a4040dcb79d824c08808dd4\",\"transient_lastDdlTime\":\"1790366424\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"7efb83138ca94380bf2d8f0f2f91c381\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790366425}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=21, eventTime=1790366425, eventType=CREATE_TABLE, dbName=default, tableName=table_127c3d62fd1c4394bcd55bc6fc18bc5d, message={"server":"","servicePrincipal":"","db":"default","table":"table_127c3d62fd1c4394bcd55bc6fc18bc5d","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_127c3d62fd1c4394bcd55bc6fc18bc5d\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790366425},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/warehouse/table_127c3d62fd1c4394bcd55bc6fc18bc5d\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_127c3d62fd1c4394bcd55bc6fc18bc5d\",\"kudu.table_id\":\"45cbddfbaced49a989c1fb61a644a713\",\"transient_lastDdlTime\":\"1790366425\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"7efb83138ca94380bf2d8f0f2f91c381\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790366425,"files":[]}, messageFormat=json-0.2, catName=hive);
I20260925 20:00:39.391844 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81d09be24f0a4687a1b17ddd02fd4e47. 1 dirs total, 0 dirs full, 0 dirs failed
*** Check failure stack trace: ***
I20260925 20:00:39.393282 5925 ts_tablet_manager.cc:1916] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 20:00:39.397241 5825 catalog_manager.cc:5142] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): alter failed for tablet dbdd993621c4410386d7914283809577 (table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]): Aborted: Op aborted
I20260925 20:00:39.398113 5825 catalog_manager.cc:5063] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): tablet dbdd993621c4410386d7914283809577 (table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]) successfully deleted
I20260925 20:00:39.398407 4644 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d0bc4cceef042389ad99a3515aa0f43 with delete_type TABLET_DATA_DELETED (replaced by tablet 465db2a7c1fd48deb8a09d3366cfc04b) from {username='slave'} at 127.0.0.1:35390
@ 0x7f459a3c0d5d google::LogMessage::Fail() at ??:0
I20260925 20:00:39.398985 4647 tablet_service.cc:1511] Processing CreateTablet for tablet 81d09be24f0a4687a1b17ddd02fd4e47 (DEFAULT_TABLE table=default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:39.399421 4647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81d09be24f0a4687a1b17ddd02fd4e47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:39.401268 5936 tablet.cc:1751] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 20:00:39.402230 5925 ts_tablet_manager.cc:1929] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:39.402359 5925 log.cc:1199] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/dbdd993621c4410386d7914283809577
I20260925 20:00:39.402817 5925 ts_tablet_manager.cc:1950] T dbdd993621c4410386d7914283809577 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:39.404620 5926 tablet_replica.cc:333] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:39.404832 5926 raft_consensus.cc:2243] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:39.405045 5926 raft_consensus.cc:2272] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:39.405659 5926 ts_tablet_manager.cc:1916] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f459a3c4b23 google::LogMessage::SendToLog() at ??:0
I20260925 20:00:39.406618 5925 tablet_replica.cc:333] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:39.406828 5925 raft_consensus.cc:2243] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:39.407004 5925 raft_consensus.cc:2272] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:39.407187 5827 catalog_manager.cc:5063] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): tablet dbdd993621c4410386d7914283809577 (table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]) successfully deleted
I20260925 20:00:39.407627 5925 ts_tablet_manager.cc:1916] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f459a3c075c google::LogMessage::Flush() at ??:0
I20260925 20:00:39.413993 4782 tablet_service.cc:1558] Processing DeleteTablet for tablet dbdd993621c4410386d7914283809577 with delete_type TABLET_DATA_DELETED (replaced by tablet 81d09be24f0a4687a1b17ddd02fd4e47) from {username='slave'} at 127.0.0.1:50064
I20260925 20:00:39.414444 4784 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d0bc4cceef042389ad99a3515aa0f43 with delete_type TABLET_DATA_DELETED (replaced by tablet 465db2a7c1fd48deb8a09d3366cfc04b) from {username='slave'} at 127.0.0.1:50064
I20260925 20:00:39.414908 4785 tablet_service.cc:1511] Processing CreateTablet for tablet 81d09be24f0a4687a1b17ddd02fd4e47 (DEFAULT_TABLE table=default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:39.415373 4785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81d09be24f0a4687a1b17ddd02fd4e47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:39.416568 5926 ts_tablet_manager.cc:1929] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:39.416721 5926 log.cc:1199] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/8d0bc4cceef042389ad99a3515aa0f43
I20260925 20:00:39.417171 5926 ts_tablet_manager.cc:1950] T 8d0bc4cceef042389ad99a3515aa0f43 P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:39.417943 5840 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58570:
name: "default.Table_d4b20390d75f45c89df2fe5d8498b337"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:39.418869 5925 ts_tablet_manager.cc:1929] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:39.419062 5925 log.cc:1199] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/8d0bc4cceef042389ad99a3515aa0f43
I20260925 20:00:39.419554 5925 ts_tablet_manager.cc:1950] T 8d0bc4cceef042389ad99a3515aa0f43 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:39.421978 5839 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58570:
name: "default.Table_ef69b46f2522492baaac95731da2bbcb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f459a3c09a1 google::LogMessage::~LogMessage() at ??:0
W20260925 20:00:39.430579 5825 catalog_manager.cc:5136] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): alter failed for tablet dbdd993621c4410386d7914283809577 (table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]),no further retry: Not found: Tablet not found: dbdd993621c4410386d7914283809577
I20260925 20:00:39.432188 5838 catalog_manager.cc:6133] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Sending DeleteTablet for 3 replicas of tablet 0827e2c7eff041369a94c31abdf9b093
I20260925 20:00:39.433123 5838 catalog_manager.cc:6934] ReplaceTablet: tablet 0827e2c7eff041369a94c31abdf9b093 deleted and replaced by tablet d56227e2824d41beb764a79713226482
I20260925 20:00:39.433878 5838 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58570:
table { table_name: "default.Table_c78f9cc1d7e74b51b29858c7d3248fb8" } new_table_name: "default.Table_1f01c68bbe404907a4d5f4d12fb9ee26" modify_external_catalogs: true
I20260925 20:00:39.435135 4506 tablet_service.cc:1558] Processing DeleteTablet for tablet 0827e2c7eff041369a94c31abdf9b093 with delete_type TABLET_DATA_DELETED (replaced by tablet d56227e2824d41beb764a79713226482) from {username='slave'} at 127.0.0.1:45194
I20260925 20:00:39.435433 4784 tablet_service.cc:1558] Processing DeleteTablet for tablet 0827e2c7eff041369a94c31abdf9b093 with delete_type TABLET_DATA_DELETED (replaced by tablet d56227e2824d41beb764a79713226482) from {username='slave'} at 127.0.0.1:50064
I20260925 20:00:39.437951 4644 tablet_service.cc:1558] Processing DeleteTablet for tablet 0827e2c7eff041369a94c31abdf9b093 with delete_type TABLET_DATA_DELETED (replaced by tablet d56227e2824d41beb764a79713226482) from {username='slave'} at 127.0.0.1:35390
I20260925 20:00:39.439404 4784 tablet_service.cc:1511] Processing CreateTablet for tablet 465db2a7c1fd48deb8a09d3366cfc04b (DEFAULT_TABLE table=default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:39.439843 4784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 465db2a7c1fd48deb8a09d3366cfc04b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:39.440074 5938 tablet_bootstrap.cc:492] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:39.440611 5825 catalog_manager.cc:5063] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): tablet 8d0bc4cceef042389ad99a3515aa0f43 (table default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d]) successfully deleted
I20260925 20:00:39.441006 4644 tablet_service.cc:1511] Processing CreateTablet for tablet 465db2a7c1fd48deb8a09d3366cfc04b (DEFAULT_TABLE table=default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:39.441393 4644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 465db2a7c1fd48deb8a09d3366cfc04b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:39.447103 5926 tablet_replica.cc:333] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:39.447144 4505 tablet_service.cc:1511] Processing CreateTablet for tablet 465db2a7c1fd48deb8a09d3366cfc04b (DEFAULT_TABLE table=default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:39.447360 5926 raft_consensus.cc:2243] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:39.447572 4505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 465db2a7c1fd48deb8a09d3366cfc04b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:39.447572 5926 raft_consensus.cc:2272] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:39.448233 5926 ts_tablet_manager.cc:1916] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:39.449295 5925 tablet_replica.cc:333] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:39.449410 5827 catalog_manager.cc:5063] TS 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): tablet 8d0bc4cceef042389ad99a3515aa0f43 (table default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d]) successfully deleted
I20260925 20:00:39.449520 5925 raft_consensus.cc:2243] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:39.449909 5925 raft_consensus.cc:2272] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:39.450552 5925 ts_tablet_manager.cc:1916] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:39.451036 5938 tablet_bootstrap.cc:654] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:39.454334 5941 tablet_bootstrap.cc:492] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:39.455004 4644 tablet_service.cc:1511] Processing CreateTablet for tablet d56227e2824d41beb764a79713226482 (DEFAULT_TABLE table=default.table_127c3d62fd1c4394bcd55bc6fc18bc5d [id=45cbddfbaced49a989c1fb61a644a713]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:39.455463 4644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d56227e2824d41beb764a79713226482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:39.456982 4784 tablet_service.cc:1511] Processing CreateTablet for tablet d56227e2824d41beb764a79713226482 (DEFAULT_TABLE table=default.table_127c3d62fd1c4394bcd55bc6fc18bc5d [id=45cbddfbaced49a989c1fb61a644a713]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:39.457401 4784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d56227e2824d41beb764a79713226482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:39.464687 5939 tablet_replica.cc:333] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:39.464903 5939 raft_consensus.cc:2243] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:39.465551 5939 pending_rounds.cc:70] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e: Trying to abort 1 pending ops.
I20260925 20:00:39.465641 5939 pending_rounds.cc:77] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7333340935352795136 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "dbdd993621c4410386d7914283809577" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.table_b74d63554f344195800ce065c2a3cddf" dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" new_extra_config { } }
I20260925 20:00:39.466297 5939 raft_consensus.cc:2272] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:39.467459 5939 ts_tablet_manager.cc:1916] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:39.470916 5926 ts_tablet_manager.cc:1929] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
@ 0x7f45b10bf2f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260925 20:00:39.471928 5940 tablet_bootstrap.cc:492] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:39.475709 4506 tablet_service.cc:1511] Processing CreateTablet for tablet d56227e2824d41beb764a79713226482 (DEFAULT_TABLE table=default.table_127c3d62fd1c4394bcd55bc6fc18bc5d [id=45cbddfbaced49a989c1fb61a644a713]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:39.476578 4506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d56227e2824d41beb764a79713226482. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f45b10bdba1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f45b10c581c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260925 20:00:39.485976 5941 tablet_bootstrap.cc:654] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:39.487144 5926 log.cc:1199] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/0827e2c7eff041369a94c31abdf9b093
I20260925 20:00:39.487684 5926 ts_tablet_manager.cc:1950] T 0827e2c7eff041369a94c31abdf9b093 P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:39.488950 5825 catalog_manager.cc:5063] TS 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): tablet 0827e2c7eff041369a94c31abdf9b093 (table default.table_127c3d62fd1c4394bcd55bc6fc18bc5d [id=45cbddfbaced49a989c1fb61a644a713]) successfully deleted
I20260925 20:00:39.489267 5940 tablet_bootstrap.cc:654] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
@ 0x7f45b10c5699 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xaaef9d std::function<>::operator()() at ??:?
I20260925 20:00:39.493944 5938 tablet_bootstrap.cc:492] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:39.494136 5938 ts_tablet_manager.cc:1403] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.056s user 0.011s sys 0.011s
I20260925 20:00:39.495187 5938 raft_consensus.cc:359] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.495434 5938 raft_consensus.cc:385] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:39.495538 5938 raft_consensus.cc:740] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:39.495824 5938 consensus_queue.cc:260] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
@ 0x7f459d110079 kudu::Thread::SuperviseThread() at ??:0
I20260925 20:00:39.496562 5938 ts_tablet_manager.cc:1434] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 20:00:39.498112 4573 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:39.498957 5925 ts_tablet_manager.cc:1929] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:39.499087 5925 log.cc:1199] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/0827e2c7eff041369a94c31abdf9b093
@ 0x7f45a07e66db start_thread at ??:0
I20260925 20:00:39.499536 5925 ts_tablet_manager.cc:1950] T 0827e2c7eff041369a94c31abdf9b093 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
@ 0x7f459926d61f clone at ??:0
I20260925 20:00:39.503711 5938 tablet_bootstrap.cc:492] T d56227e2824d41beb764a79713226482 P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:39.506001 5938 tablet_bootstrap.cc:654] T d56227e2824d41beb764a79713226482 P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:39.510761 5941 tablet_bootstrap.cc:492] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:39.510993 5941 ts_tablet_manager.cc:1403] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.057s user 0.002s sys 0.005s
I20260925 20:00:39.511245 5940 tablet_bootstrap.cc:492] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:39.511415 5940 ts_tablet_manager.cc:1403] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.040s user 0.000s sys 0.007s
I20260925 20:00:39.512128 5941 raft_consensus.cc:359] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.512429 5941 raft_consensus.cc:385] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:39.512337 5940 raft_consensus.cc:359] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.512538 5941 raft_consensus.cc:740] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:39.512547 5940 raft_consensus.cc:385] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:39.512646 5940 raft_consensus.cc:740] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:39.512854 5941 consensus_queue.cc:260] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.513645 5941 ts_tablet_manager.cc:1434] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 20:00:39.514043 5941 tablet_bootstrap.cc:492] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:39.518481 5939 ts_tablet_manager.cc:1929] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:39.518671 5939 log.cc:1199] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/dbdd993621c4410386d7914283809577
I20260925 20:00:39.519287 5939 ts_tablet_manager.cc:1950] T dbdd993621c4410386d7914283809577 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:39.520744 5939 tablet_replica.cc:333] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:39.521078 5939 raft_consensus.cc:2243] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:39.521492 5939 raft_consensus.cc:2272] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:39.513019 5940 consensus_queue.cc:260] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.522365 5939 ts_tablet_manager.cc:1916] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:39.525048 5941 tablet_bootstrap.cc:654] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:39.527515 5938 tablet_bootstrap.cc:492] T d56227e2824d41beb764a79713226482 P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:39.527774 5938 ts_tablet_manager.cc:1403] T d56227e2824d41beb764a79713226482 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.024s user 0.012s sys 0.011s
I20260925 20:00:39.528718 5940 ts_tablet_manager.cc:1434] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.017s user 0.000s sys 0.003s
I20260925 20:00:39.528769 5938 raft_consensus.cc:359] T d56227e2824d41beb764a79713226482 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.529093 5938 raft_consensus.cc:385] T d56227e2824d41beb764a79713226482 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:39.529203 5938 raft_consensus.cc:740] T d56227e2824d41beb764a79713226482 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:39.529541 5938 consensus_queue.cc:260] T d56227e2824d41beb764a79713226482 P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.530123 5938 ts_tablet_manager.cc:1434] T d56227e2824d41beb764a79713226482 P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 20:00:39.531926 5938 tablet_bootstrap.cc:492] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:39.533141 5939 ts_tablet_manager.cc:1929] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:39.533313 5939 log.cc:1199] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/8d0bc4cceef042389ad99a3515aa0f43
I20260925 20:00:39.533424 5940 tablet_bootstrap.cc:492] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:39.533754 5939 ts_tablet_manager.cc:1950] T 8d0bc4cceef042389ad99a3515aa0f43 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:39.533972 5938 tablet_bootstrap.cc:654] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:39.535161 5939 tablet_replica.cc:333] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:39.535264 5940 tablet_bootstrap.cc:654] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:39.535480 5939 raft_consensus.cc:2243] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:39.535748 5939 raft_consensus.cc:2272] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:39.542366 5939 ts_tablet_manager.cc:1916] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:39.548527 5806 raft_consensus.cc:493] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:39.548714 5806 raft_consensus.cc:515] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.549889 5806 leader_election.cc:290] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:39.550871 4524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d09be24f0a4687a1b17ddd02fd4e47" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:39.551251 4524 raft_consensus.cc:2468] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:39.552038 4741 leader_election.cc:304] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:39.552424 5806 raft_consensus.cc:2804] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:39.552538 5806 raft_consensus.cc:493] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:39.552614 5806 raft_consensus.cc:3060] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:39.555505 4665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d09be24f0a4687a1b17ddd02fd4e47" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:39.555852 4665 raft_consensus.cc:2468] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:39.556134 5806 raft_consensus.cc:515] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.557547 4524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d09be24f0a4687a1b17ddd02fd4e47" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:39.557885 4524 raft_consensus.cc:3060] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:39.558013 5806 leader_election.cc:290] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Requested vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:39.558128 4665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d09be24f0a4687a1b17ddd02fd4e47" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:39.558348 4665 raft_consensus.cc:3060] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:39.561168 4524 raft_consensus.cc:2468] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:39.561533 4665 raft_consensus.cc:2468] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:39.561878 4741 leader_election.cc:304] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:39.562728 5806 raft_consensus.cc:2804] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:39.563342 5939 ts_tablet_manager.cc:1929] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:39.563478 5939 log.cc:1199] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/0827e2c7eff041369a94c31abdf9b093
I20260925 20:00:39.564079 5939 ts_tablet_manager.cc:1950] T 0827e2c7eff041369a94c31abdf9b093 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:39.575346 5941 tablet_bootstrap.cc:492] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:39.575666 5941 ts_tablet_manager.cc:1403] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.062s user 0.023s sys 0.016s
I20260925 20:00:39.576138 5940 tablet_bootstrap.cc:492] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:39.576382 5940 ts_tablet_manager.cc:1403] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.043s user 0.004s sys 0.000s
I20260925 20:00:39.576793 5941 raft_consensus.cc:359] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.577085 5941 raft_consensus.cc:385] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:39.577186 5941 raft_consensus.cc:740] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:39.577697 5940 raft_consensus.cc:359] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.577945 5940 raft_consensus.cc:385] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:39.578037 5940 raft_consensus.cc:740] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:39.578073 5941 consensus_queue.cc:260] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.578444 5940 consensus_queue.cc:260] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.584127 5940 ts_tablet_manager.cc:1434] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260925 20:00:39.585834 5941 ts_tablet_manager.cc:1434] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.010s user 0.005s sys 0.001s
I20260925 20:00:39.588244 5941 tablet_bootstrap.cc:492] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:39.588837 5940 tablet_bootstrap.cc:492] T d56227e2824d41beb764a79713226482 P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:39.590611 5940 tablet_bootstrap.cc:654] T d56227e2824d41beb764a79713226482 P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:39.590739 5941 tablet_bootstrap.cc:654] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:39.591943 5806 raft_consensus.cc:697] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Becoming Leader. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Running, Role: LEADER
I20260925 20:00:39.592409 5806 consensus_queue.cc:237] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.593241 5938 tablet_bootstrap.cc:492] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:39.593405 5938 ts_tablet_manager.cc:1403] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.062s user 0.013s sys 0.040s
I20260925 20:00:39.594586 5938 raft_consensus.cc:359] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.594786 5938 raft_consensus.cc:385] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:39.595216 5938 raft_consensus.cc:740] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:39.595664 5938 consensus_queue.cc:260] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.601182 5940 tablet_bootstrap.cc:492] T d56227e2824d41beb764a79713226482 P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:39.601394 5940 ts_tablet_manager.cc:1403] T d56227e2824d41beb764a79713226482 P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.002s
I20260925 20:00:39.601461 5938 ts_tablet_manager.cc:1434] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.008s user 0.001s sys 0.002s
I20260925 20:00:39.605010 5940 raft_consensus.cc:359] T d56227e2824d41beb764a79713226482 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.605242 5940 raft_consensus.cc:385] T d56227e2824d41beb764a79713226482 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:39.605335 5940 raft_consensus.cc:740] T d56227e2824d41beb764a79713226482 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:39.605611 5940 consensus_queue.cc:260] T d56227e2824d41beb764a79713226482 P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.606827 5941 tablet_bootstrap.cc:492] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e: No bootstrap required, opened a new log
I20260925 20:00:39.607010 5941 ts_tablet_manager.cc:1403] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.004s
I20260925 20:00:39.607944 5941 raft_consensus.cc:359] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.608145 5941 raft_consensus.cc:385] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:39.608237 5941 raft_consensus.cc:740] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:39.608506 5941 consensus_queue.cc:260] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.608661 5940 ts_tablet_manager.cc:1434] T d56227e2824d41beb764a79713226482 P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.007s user 0.001s sys 0.002s
I20260925 20:00:39.609202 5941 ts_tablet_manager.cc:1434] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 20:00:39.627935 5806 raft_consensus.cc:493] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:39.628153 5806 raft_consensus.cc:515] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.628865 5806 leader_election.cc:290] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:39.629447 4665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "465db2a7c1fd48deb8a09d3366cfc04b" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:39.629420 4524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "465db2a7c1fd48deb8a09d3366cfc04b" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:39.629637 4665 raft_consensus.cc:2468] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:39.629650 4524 raft_consensus.cc:2468] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:39.630128 4741 leader_election.cc:304] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:39.630461 5806 raft_consensus.cc:2804] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:39.630568 5806 raft_consensus.cc:493] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:39.630667 5806 raft_consensus.cc:3060] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:39.633648 5806 raft_consensus.cc:515] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.634868 4665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "465db2a7c1fd48deb8a09d3366cfc04b" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:39.635058 4665 raft_consensus.cc:3060] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:39.635360 5806 leader_election.cc:290] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Requested vote from peers 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647), 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:39.635480 4524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "465db2a7c1fd48deb8a09d3366cfc04b" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:39.635671 4524 raft_consensus.cc:3060] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:39.638401 4665 raft_consensus.cc:2468] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:39.638839 4524 raft_consensus.cc:2468] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:39.639058 4739 leader_election.cc:304] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:39.639628 5806 raft_consensus.cc:2804] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:39.640201 5806 raft_consensus.cc:697] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Becoming Leader. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Running, Role: LEADER
I20260925 20:00:39.640513 5806 consensus_queue.cc:237] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.671187 5806 raft_consensus.cc:493] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:39.671437 5806 raft_consensus.cc:515] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.672597 5806 leader_election.cc:290] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:39.672719 4524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56227e2824d41beb764a79713226482" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:39.672753 4665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56227e2824d41beb764a79713226482" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0" is_pre_election: true
I20260925 20:00:39.673022 4665 raft_consensus.cc:2468] T d56227e2824d41beb764a79713226482 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:39.673022 4524 raft_consensus.cc:2468] T d56227e2824d41beb764a79713226482 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 0.
I20260925 20:00:39.673543 4741 leader_election.cc:304] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:39.673868 5806 raft_consensus.cc:2804] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:39.673992 5806 raft_consensus.cc:493] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:39.674081 5806 raft_consensus.cc:3060] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:39.677181 5806 raft_consensus.cc:515] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:39.678030 5806 leader_election.cc:290] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Requested vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:39.678567 4665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56227e2824d41beb764a79713226482" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a6f571dfa749b7905d8458d4c929b0"
I20260925 20:00:39.678563 4524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56227e2824d41beb764a79713226482" candidate_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:39.678789 4524 raft_consensus.cc:3060] T d56227e2824d41beb764a79713226482 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:39.678798 4665 raft_consensus.cc:3060] T d56227e2824d41beb764a79713226482 P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 20:00:39.680976 4665 raft_consensus.cc:2468] T d56227e2824d41beb764a79713226482 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:39.681500 4739 leader_election.cc:304] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46a6f571dfa749b7905d8458d4c929b0, dd2623c3b2c3497ca1df4a676fa6807e; no voters:
I20260925 20:00:39.681818 5806 raft_consensus.cc:2804] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:39.681972 4524 raft_consensus.cc:2468] T d56227e2824d41beb764a79713226482 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2623c3b2c3497ca1df4a676fa6807e in term 1.
I20260925 20:00:39.681993 5806 raft_consensus.cc:697] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Becoming Leader. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Running, Role: LEADER
I20260925 20:00:39.682523 5806 consensus_queue.cc:237] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
W20260925 20:00:39.770531 4738 connection.cc:573] server connection from 127.0.0.1:50064 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:39.770550 4457 connection.cc:573] server connection from 127.0.0.1:45194 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:39.770817 4456 connection.cc:573] client connection to 127.3.199.253:41897 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:39.771310 4602 connection.cc:573] client connection to 127.3.199.253:41897 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:39.771981 4862 connection.cc:573] client connection to 127.3.199.253:41897 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:39.773109 4862 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.3.199.253:41897)
W20260925 20:00:39.773910 4862 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.3.199.253:41897)
W20260925 20:00:39.774088 4862 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.199.253:41897)
W20260925 20:00:39.774222 4862 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.199.253:41897)
I20260925 20:00:39.947172 5967 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:39.947417 5967 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:39.948243 5967 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:39.950387 5700 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:39.950565 5701 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:39.950703 5701 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:39.950997 5967 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260925 20:00:39.995707 5676 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:00:39.995965 5676 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:00:40.377616 5806 consensus_queue.cc:1048] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:40.387600 5975 consensus_queue.cc:1048] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 20:00:40.631207 5981 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:40.631445 5981 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:40.632126 5981 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:40.633442 5701 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260925 20:00:40.634853 5701 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:40.635308 5700 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:40.635478 5700 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:40.635772 5981 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260925 20:00:40.702217 5806 consensus_queue.cc:1048] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:40.708040 5972 consensus_queue.cc:1048] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:40.751900 5972 consensus_queue.cc:1048] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:40.757620 5972 consensus_queue.cc:1048] T d56227e2824d41beb764a79713226482 P dd2623c3b2c3497ca1df4a676fa6807e [LEADER]: Connected to new peer: Peer: permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:41.281832 3871 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu pid 5676
I20260925 20:00:41.289296 5744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 20:00:41.406585 3871 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu with pid 5676
I20260925 20:00:42.392537 3871 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
/tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.254:41289
--webserver_interface=127.3.199.254
--webserver_port=32859
--builtin_ntp_servers=127.3.199.212:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260925 20:00:42.768807 6011 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 20:00:42.769315 6011 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 20:00:42.769510 6011 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 20:00:42.779184 6011 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 20:00:42.779299 6011 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 20:00:42.779385 6011 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 20:00:42.779495 6011 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 20:00:42.791060 6011 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.199.212:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.254:41289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.199.254
--webserver_port=32859
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 25 Sep 2026 19:43:21 UTC on bdcb31816ec0
build id 13997
ASAN enabled
I20260925 20:00:42.793092 6011 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 20:00:42.795326 6011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 20:00:42.805132 6019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:42.805145 6022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:42.805231 6020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 20:00:42.806317 6011 server_base.cc:1061] running on GCE node
I20260925 20:00:42.807271 6011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 20:00:42.809068 6011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 20:00:42.810400 6011 hybrid_clock.cc:648] HybridClock initialized: now 1790366442810318 us; error 76 us; skew 500 ppm
I20260925 20:00:42.811018 6011 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 20:00:42.813503 6011 webserver.cc:533] Webserver started at http://127.3.199.254:32859/ using document root <none> and password file <none>
I20260925 20:00:42.814311 6011 fs_manager.cc:362] Metadata directory not provided
I20260925 20:00:42.814405 6011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 20:00:42.821640 6011 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260925 20:00:42.825678 6028 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:42.827646 6011 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260925 20:00:42.827877 6011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data,/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
uuid: "169b6de20560440b99cc08af549ce477"
format_stamp: "Formatted at 2026-09-25 20:00:16 on dist-test-slave-9gwk"
I20260925 20:00:42.828878 6011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 20:00:42.860046 6011 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 20:00:42.861195 6011 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 20:00:42.861709 6011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 20:00:42.892077 6011 rpc_server.cc:307] RPC server started. Bound to: 127.3.199.254:41289
I20260925 20:00:42.892077 6079 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.199.254:41289 every 8 connection(s)
I20260925 20:00:42.894503 6011 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-0/data/info.pb
I20260925 20:00:42.896425 3871 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu as pid 6011
I20260925 20:00:42.908501 6080 sys_catalog.cc:263] Verifying existing consensus state
I20260925 20:00:42.913149 6080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Bootstrap starting.
I20260925 20:00:42.934454 6080 log.cc:826] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Log is configured to *not* fsync() on all Append() calls
I20260925 20:00:43.011541 6080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Bootstrap replayed 1/1 log segments. Stats: ops{read=99 overwritten=0 applied=99 ignored=0} inserts{seen=58 ignored=0} mutations{seen=81 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 20:00:43.012169 6080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: Bootstrap complete.
I20260925 20:00:43.017866 6080 raft_consensus.cc:359] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:43.018600 6080 raft_consensus.cc:740] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: 169b6de20560440b99cc08af549ce477, State: Initialized, Role: FOLLOWER
I20260925 20:00:43.019461 6080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 99, Last appended: 8.99, Last appended by leader: 99, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:43.020395 6084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:43.020707 6084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:43.021535 6080 sys_catalog.cc:565] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 20:00:43.034641 6098 catalog_manager.cc:1308] Loaded cluster ID: 7efb83138ca94380bf2d8f0f2f91c381
I20260925 20:00:43.034869 6098 catalog_manager.cc:1601] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: loading cluster ID for follower catalog manager: success
I20260925 20:00:43.039873 6098 catalog_manager.cc:1623] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: acquiring CA information for follower catalog manager: success
I20260925 20:00:43.043684 6098 catalog_manager.cc:1651] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 20:00:43.054977 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:00:43.059295 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:00:43.181670 4851 heartbeater.cc:344] Connected to a master server at 127.3.199.254:41289
I20260925 20:00:43.185658 6045 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" instance_seqno: 1790366420065491) as {username='slave'} at 127.3.199.195:50409; Asking this server to re-register.
I20260925 20:00:43.186656 4851 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:43.189242 6045 ts_manager.cc:194] Registered new tserver with Master: dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:43.281922 4713 heartbeater.cc:344] Connected to a master server at 127.3.199.254:41289
I20260925 20:00:43.283979 6045 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" instance_seqno: 1790366419525613) as {username='slave'} at 127.3.199.194:45151; Asking this server to re-register.
I20260925 20:00:43.284823 4713 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:43.286957 4572 heartbeater.cc:344] Connected to a master server at 127.3.199.254:41289
I20260925 20:00:43.287505 6045 ts_manager.cc:194] Registered new tserver with Master: 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:43.289108 6045 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" instance_seqno: 1790366418849966) as {username='slave'} at 127.3.199.193:47945; Asking this server to re-register.
I20260925 20:00:43.289840 4572 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:43.292120 6045 ts_manager.cc:194] Registered new tserver with Master: 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:43.530019 6107 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:43.530383 6107 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:43.532830 6107 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:43.533758 6032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:43.535734 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:43.536007 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:43.536235 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:43.536777 6107 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260925 20:00:44.149567 6112 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:44.149806 6112 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:44.150677 6112 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:44.153971 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:44.154199 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:44.154424 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:44.154824 6112 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260925 20:00:44.880671 6124 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:44.880896 6124 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:44.881685 6124 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:44.884378 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:44.884558 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:44.884702 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:44.885008 6124 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260925 20:00:46.077409 3871 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
/tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.253:41897
--webserver_interface=127.3.199.253
--webserver_port=41541
--builtin_ntp_servers=127.3.199.212:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 20:00:46.395287 6143 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:46.395533 6143 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:46.396265 6143 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:46.398753 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:46.399449 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:46.399586 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:46.399936 6143 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260925 20:00:46.444429 6138 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 20:00:46.444929 6138 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 20:00:46.445065 6138 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 20:00:46.454109 6138 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 20:00:46.454226 6138 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 20:00:46.454314 6138 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 20:00:46.454411 6138 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 20:00:46.466586 6138 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.199.212:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.253:41897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.199.253
--webserver_port=41541
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 25 Sep 2026 19:43:21 UTC on bdcb31816ec0
build id 13997
ASAN enabled
I20260925 20:00:46.469063 6138 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 20:00:46.471160 6138 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 20:00:46.481115 6151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:46.482647 6152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:46.483883 6154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 20:00:46.484597 6138 server_base.cc:1061] running on GCE node
I20260925 20:00:46.485623 6138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 20:00:46.487685 6138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 20:00:46.489063 6138 hybrid_clock.cc:648] HybridClock initialized: now 1790366446488984 us; error 76 us; skew 500 ppm
I20260925 20:00:46.489602 6138 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 20:00:46.492915 6138 webserver.cc:533] Webserver started at http://127.3.199.253:41541/ using document root <none> and password file <none>
I20260925 20:00:46.493854 6138 fs_manager.cc:362] Metadata directory not provided
I20260925 20:00:46.493963 6138 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 20:00:46.500615 6138 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.007s sys 0.000s
I20260925 20:00:46.503928 6160 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:46.505858 6138 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 20:00:46.506039 6138 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data,/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
uuid: "da29ee550f82456fa7c5ce7ea916909f"
format_stamp: "Formatted at 2026-09-25 20:00:17 on dist-test-slave-9gwk"
I20260925 20:00:46.506842 6138 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 20:00:46.557291 6138 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 20:00:46.558437 6138 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 20:00:46.558761 6138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 20:00:46.588414 6138 rpc_server.cc:307] RPC server started. Bound to: 127.3.199.253:41897
I20260925 20:00:46.588511 6212 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.199.253:41897 every 8 connection(s)
I20260925 20:00:46.590205 6138 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
I20260925 20:00:46.591135 3871 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu as pid 6138
I20260925 20:00:46.600204 6213 sys_catalog.cc:263] Verifying existing consensus state
I20260925 20:00:46.604918 6213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Bootstrap starting.
I20260925 20:00:46.623963 6213 log.cc:826] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Log is configured to *not* fsync() on all Append() calls
I20260925 20:00:46.693094 4852 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:46.705761 6213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Bootstrap replayed 1/1 log segments. Stats: ops{read=99 overwritten=0 applied=99 ignored=0} inserts{seen=58 ignored=0} mutations{seen=81 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 20:00:46.706558 6213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Bootstrap complete.
I20260925 20:00:46.714244 6213 raft_consensus.cc:359] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 8 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:46.714929 6213 raft_consensus.cc:740] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: da29ee550f82456fa7c5ce7ea916909f, State: Initialized, Role: FOLLOWER
I20260925 20:00:46.715623 6213 consensus_queue.cc:260] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 99, Last appended: 8.99, Last appended by leader: 99, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:46.716526 6219 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:46.716842 6219 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:46.717872 6213 sys_catalog.cc:565] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 20:00:46.744467 6233 catalog_manager.cc:1308] Loaded cluster ID: 7efb83138ca94380bf2d8f0f2f91c381
I20260925 20:00:46.744611 6233 catalog_manager.cc:1601] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: loading cluster ID for follower catalog manager: success
I20260925 20:00:46.750190 6233 catalog_manager.cc:1623] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: acquiring CA information for follower catalog manager: success
I20260925 20:00:46.753583 6233 catalog_manager.cc:1651] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 20:00:46.766017 6138 master_runner.cc:187] Error getting master registration for 127.3.199.252:37957: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111),
I20260925 20:00:46.771181 6138 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:00:46.788522 4573 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:46.788784 4573 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:46.792434 6178 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" instance_seqno: 1790366418849966) as {username='slave'} at 127.3.199.193:45099; Asking this server to re-register.
I20260925 20:00:46.793524 4573 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:46.794339 4714 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:46.795742 6178 ts_manager.cc:194] Registered new tserver with Master: 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:46.796906 6177 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" instance_seqno: 1790366419525613) as {username='slave'} at 127.3.199.194:50731; Asking this server to re-register.
I20260925 20:00:46.797866 4714 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:46.799890 6177 ts_manager.cc:194] Registered new tserver with Master: 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:47.199168 6177 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" instance_seqno: 1790366420065491) as {username='slave'} at 127.3.199.195:52241; Asking this server to re-register.
I20260925 20:00:47.200018 4852 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:47.201974 6177 ts_manager.cc:194] Registered new tserver with Master: dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:47.264160 6243 raft_consensus.cc:493] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:47.264570 6243 raft_consensus.cc:515] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:47.266636 6243 leader_election.cc:290] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:47.267444 6055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da29ee550f82456fa7c5ce7ea916909f" candidate_term: 9 candidate_status { last_received { term: 8 index: 99 } } ignore_live_leader: false dest_uuid: "169b6de20560440b99cc08af549ce477" is_pre_election: true
W20260925 20:00:47.267656 6164 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:47.267834 6055 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da29ee550f82456fa7c5ce7ea916909f in term 8.
I20260925 20:00:47.268644 6163 leader_election.cc:304] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 169b6de20560440b99cc08af549ce477, da29ee550f82456fa7c5ce7ea916909f; no voters:
I20260925 20:00:47.269073 6243 raft_consensus.cc:2804] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260925 20:00:47.269304 6243 raft_consensus.cc:493] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:47.269454 6243 raft_consensus.cc:3060] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 8 FOLLOWER]: Advancing to term 9
W20260925 20:00:47.269558 6164 leader_election.cc:336] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:47.273489 6243 raft_consensus.cc:515] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:47.274204 6243 leader_election.cc:290] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 9 election: Requested vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:47.274907 6055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da29ee550f82456fa7c5ce7ea916909f" candidate_term: 9 candidate_status { last_received { term: 8 index: 99 } } ignore_live_leader: false dest_uuid: "169b6de20560440b99cc08af549ce477"
I20260925 20:00:47.275166 6055 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 8 FOLLOWER]: Advancing to term 9
W20260925 20:00:47.275784 6164 leader_election.cc:336] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:47.279042 6055 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da29ee550f82456fa7c5ce7ea916909f in term 9.
I20260925 20:00:47.279561 6163 leader_election.cc:304] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 169b6de20560440b99cc08af549ce477, da29ee550f82456fa7c5ce7ea916909f; no voters: 7f43a6a14ef547f2aeb22ce2b334383e
I20260925 20:00:47.279968 6243 raft_consensus.cc:2804] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 9 FOLLOWER]: Leader election won for term 9
I20260925 20:00:47.280552 6243 raft_consensus.cc:697] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 9 LEADER]: Becoming Leader. State: Replica: da29ee550f82456fa7c5ce7ea916909f, State: Running, Role: LEADER
I20260925 20:00:47.281138 6243 consensus_queue.cc:237] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 99, Committed index: 99, Last appended: 8.99, Last appended by leader: 99, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:47.283740 6246 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: New leader da29ee550f82456fa7c5ce7ea916909f. Latest consensus state: current_term: 9 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:47.283982 6246 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:47.284972 6248 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260925 20:00:47.288614 6248 catalog_manager.cc:692] Loaded metadata for table default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330]
I20260925 20:00:47.289067 6248 catalog_manager.cc:692] Loaded metadata for table default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf]
I20260925 20:00:47.289413 6248 catalog_manager.cc:692] Loaded metadata for table default.table_703616ef44ba43f9afb9c4df73eb372c [id=25f6e1ade43c4519bed519037ee8f761]
I20260925 20:00:47.289742 6248 catalog_manager.cc:692] Loaded metadata for table default.table_667eace0e712438bbd1211f7e3e665f8 [id=439b3d65790141d1abfbca4788836f80]
I20260925 20:00:47.290045 6248 catalog_manager.cc:692] Loaded metadata for table default.table_127c3d62fd1c4394bcd55bc6fc18bc5d [id=45cbddfbaced49a989c1fb61a644a713]
I20260925 20:00:47.290341 6248 catalog_manager.cc:692] Loaded metadata for table default.table_2ebd7ecb76b44d13a6847408f7fd7d17 [id=4b80e3afa6954e2ea8b6d1b5ec79a9d1]
I20260925 20:00:47.290642 6248 catalog_manager.cc:692] Loaded metadata for table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]
I20260925 20:00:47.290973 6248 catalog_manager.cc:692] Loaded metadata for table default.table_c78f9cc1d7e74b51b29858c7d3248fb8 [id=7ead82097c49448f85b1dbef027be066]
I20260925 20:00:47.291265 6248 catalog_manager.cc:692] Loaded metadata for table default.table_098b200909f842c88a38f11192f4dc25 [id=86f1c8a16cc14ef9b72c95060313b3ad]
I20260925 20:00:47.291569 6248 catalog_manager.cc:692] Loaded metadata for table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]
I20260925 20:00:47.291882 6248 catalog_manager.cc:692] Loaded metadata for table default.table_5b8386185e8e4fcd99110bd5137a578c [id=a385e0b7db6a49e8b1baf34a5fdd3473]
I20260925 20:00:47.292273 6248 catalog_manager.cc:692] Loaded metadata for table default.table_7722a249d10f4616ba5b5bb2fc149c18 [id=a8f568d7faab469ba7ea8a2a38bfd638]
I20260925 20:00:47.292783 6248 catalog_manager.cc:692] Loaded metadata for table default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d]
I20260925 20:00:47.293224 6248 catalog_manager.cc:692] Loaded metadata for table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]
I20260925 20:00:47.293762 6248 catalog_manager.cc:692] Loaded metadata for table default.table_08807903f9884f2a9707d4066e7f2f52 [id=f6fa165ad5e7471589cb63403548ab2c]
I20260925 20:00:47.296583 6248 tablet_loader.cc:96] loaded metadata for tablet 2ddb9fbd0aeb4c4a985508ac67c93227 (table default.table_7722a249d10f4616ba5b5bb2fc149c18 [id=a8f568d7faab469ba7ea8a2a38bfd638])
I20260925 20:00:47.296958 6248 tablet_loader.cc:96] loaded metadata for tablet 44f42691903d4e2bb3f24ecb75a3f86e (table default.table_08807903f9884f2a9707d4066e7f2f52 [id=f6fa165ad5e7471589cb63403548ab2c])
I20260925 20:00:47.297554 6248 tablet_loader.cc:96] loaded metadata for tablet 465db2a7c1fd48deb8a09d3366cfc04b (table default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d])
I20260925 20:00:47.297955 6248 tablet_loader.cc:96] loaded metadata for tablet 5982cfa34e4e40d3894cbf46aab33715 (table default.table_2ebd7ecb76b44d13a6847408f7fd7d17 [id=4b80e3afa6954e2ea8b6d1b5ec79a9d1])
I20260925 20:00:47.298377 6248 tablet_loader.cc:96] loaded metadata for tablet 59a14370409146a1847c5b76755e3feb (table default.table_c78f9cc1d7e74b51b29858c7d3248fb8 [id=7ead82097c49448f85b1dbef027be066])
I20260925 20:00:47.299162 6248 tablet_loader.cc:96] loaded metadata for tablet 653a4121e8934471b99a9853b73617ab (table default.table_5b8386185e8e4fcd99110bd5137a578c [id=a385e0b7db6a49e8b1baf34a5fdd3473])
I20260925 20:00:47.299598 6248 tablet_loader.cc:96] loaded metadata for tablet 72c3c64108f348a5ab893e6954d72d77 (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812])
I20260925 20:00:47.300019 6248 tablet_loader.cc:96] loaded metadata for tablet 81d09be24f0a4687a1b17ddd02fd4e47 (table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22])
I20260925 20:00:47.300523 6248 tablet_loader.cc:96] loaded metadata for tablet 83c0e7bd6ef040b9af169d9677137836 (table default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330])
I20260925 20:00:47.301532 6248 tablet_loader.cc:96] loaded metadata for tablet b321f36472cc42babf1ec3f64ab990ed (table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7])
I20260925 20:00:47.301882 6248 tablet_loader.cc:96] loaded metadata for tablet b59f74f459c542a88e8bef5f1c51357d (table default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf])
I20260925 20:00:47.302313 6248 tablet_loader.cc:96] loaded metadata for tablet b65400989b944ca988d6f26838277336 (table default.table_667eace0e712438bbd1211f7e3e665f8 [id=439b3d65790141d1abfbca4788836f80])
I20260925 20:00:47.302738 6248 tablet_loader.cc:96] loaded metadata for tablet d56227e2824d41beb764a79713226482 (table default.table_127c3d62fd1c4394bcd55bc6fc18bc5d [id=45cbddfbaced49a989c1fb61a644a713])
I20260925 20:00:47.304076 6248 tablet_loader.cc:96] loaded metadata for tablet e0648fcd521e4db19359c743447246b7 (table default.table_703616ef44ba43f9afb9c4df73eb372c [id=25f6e1ade43c4519bed519037ee8f761])
I20260925 20:00:47.304533 6248 tablet_loader.cc:96] loaded metadata for tablet ee843d55757347ef9942dffd917c0ff8 (table default.table_098b200909f842c88a38f11192f4dc25 [id=86f1c8a16cc14ef9b72c95060313b3ad])
I20260925 20:00:47.305243 6248 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260925 20:00:47.307022 6248 catalog_manager.cc:1308] Loaded cluster ID: 7efb83138ca94380bf2d8f0f2f91c381
I20260925 20:00:47.307121 6248 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260925 20:00:47.308732 6248 catalog_manager.cc:1553] Loading token signing keys...
I20260925 20:00:47.309772 6248 catalog_manager.cc:6230] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Loaded TSK: 0
I20260925 20:00:47.310678 6248 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260925 20:00:47.311725 6248 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260925 20:00:47.313014 6248 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: 20
I20260925 20:00:47.434051 6177 master_service.cc:960] ReplaceTablet: received request to replace tablet b65400989b944ca988d6f26838277336 from {username='slave'} at 127.0.0.1:36054
I20260925 20:00:47.434146 6178 master_service.cc:960] ReplaceTablet: received request to replace tablet 465db2a7c1fd48deb8a09d3366cfc04b from {username='slave'} at 127.0.0.1:36054
I20260925 20:00:47.434672 6174 master_service.cc:960] ReplaceTablet: received request to replace tablet 72c3c64108f348a5ab893e6954d72d77 from {username='slave'} at 127.0.0.1:36054
I20260925 20:00:47.435232 6175 master_service.cc:960] ReplaceTablet: received request to replace tablet 72c3c64108f348a5ab893e6954d72d77 from {username='slave'} at 127.0.0.1:36054
I20260925 20:00:47.435674 6176 master_service.cc:960] ReplaceTablet: received request to replace tablet e0648fcd521e4db19359c743447246b7 from {username='slave'} at 127.0.0.1:36054
I20260925 20:00:47.436757 6173 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36054:
table { table_name: "default.Table_c860d14624744708bf4ff50a84517b59" } modify_external_catalogs: true
I20260925 20:00:47.437053 6173 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36054:
table { table_name: "default.Table_c860d14624744708bf4ff50a84517b59" } modify_external_catalogs: true
I20260925 20:00:47.437768 6172 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36054:
table { table_name: "default.Table_098b200909f842c88a38f11192f4dc25" } new_table_name: "default.Table_12cc527a0517451f9a6f226194c4ca68" modify_external_catalogs: true
I20260925 20:00:47.438977 6169 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36054:
table { table_name: "default.Table_b74d63554f344195800ce065c2a3cddf" } new_table_name: "default.Table_1f3deb6e388f4abf83c97af376b6cf64" modify_external_catalogs: true
I20260925 20:00:47.438727 6171 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36054:
name: "default.Table_6796b08b82a249d8b499df979ebeb52a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:47.439744 6170 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36054:
name: "default.Table_f9d13749640c4f11913f7449b48cf6c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:47.443559 6055 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 9 FOLLOWER]: Refusing update from remote peer da29ee550f82456fa7c5ce7ea916909f: Log matching property violated. Preceding OpId in replica: term: 8 index: 99. Preceding OpId from leader: term: 9 index: 101. (index mismatch)
W20260925 20:00:47.443625 6164 consensus_peers.cc:597] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f -> Peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Couldn't send request to peer 7f43a6a14ef547f2aeb22ce2b334383e. Status: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 20:00:47.444757 6246 consensus_queue.cc:1048] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [LEADER]: Connected to new peer: Peer: permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 100, Last known committed idx: 99, Time since last communication: 0.000s
F20260925 20:00:47.447556 6232 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 21, previous 21; dumping first 2 out of 13 received events: NotificationEvent(eventId=21, eventTime=1790366425, eventType=DROP_TABLE, dbName=default, tableName=table_30a6c0d501b04bfc882cec2f43c06f83, message={"server":"","servicePrincipal":"","db":"default","table":"table_30a6c0d501b04bfc882cec2f43c06f83","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_30a6c0d501b04bfc882cec2f43c06f83\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790366424},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/warehouse/table_30a6c0d501b04bfc882cec2f43c06f83\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_30a6c0d501b04bfc882cec2f43c06f83\",\"kudu.table_id\":\"44287e882a4040dcb79d824c08808dd4\",\"transient_lastDdlTime\":\"1790366424\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"7efb83138ca94380bf2d8f0f2f91c381\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790366425}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=21, eventTime=1790366425, eventType=CREATE_TABLE, dbName=default, tableName=table_127c3d62fd1c4394bcd55bc6fc18bc5d, message={"server":"","servicePrincipal":"","db":"default","table":"table_127c3d62fd1c4394bcd55bc6fc18bc5d","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_127c3d62fd1c4394bcd55bc6fc18bc5d\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790366425},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/warehouse/table_127c3d62fd1c4394bcd55bc6fc18bc5d\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_127c3d62fd1c4394bcd55bc6fc18bc5d\",\"kudu.table_id\":\"45cbddfbaced49a989c1fb61a644a713\",\"transient_lastDdlTime\":\"1790366425\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"7efb83138ca94380bf2d8f0f2f91c381\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790366425,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f6c3dbe9d5d google::LogMessage::Fail() at ??:0
@ 0x7f6c3dbedb23 google::LogMessage::SendToLog() at ??:0
I20260925 20:00:47.452019 6250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da29ee550f82456fa7c5ce7ea916909f. Latest consensus state: current_term: 9 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:47.452419 6250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f6c3dbe975c google::LogMessage::Flush() at ??:0
I20260925 20:00:47.453971 6249 mvcc.cc:204] Tried to move back new op lower bound from 7333340968714711040 to 7333340968067633152. Current Snapshot: MvccSnapshot[applied={T|T < 7333340968714711040}]
@ 0x7f6c3dbe99a1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f6c548e82f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260925 20:00:47.469409 6246 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 9 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:47.469691 6246 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:47.469935 6174 catalog_manager.cc:6133] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Sending DeleteTablet for 3 replicas of tablet 72c3c64108f348a5ab893e6954d72d77
I20260925 20:00:47.473124 6175 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36054:
table { table_name: "default.Table_08807903f9884f2a9707d4066e7f2f52" } new_table_name: "default.Table_6ea0804544d84afca71f1f239845be69" modify_external_catalogs: true
I20260925 20:00:47.473286 6250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:47.473489 6250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:47.473690 6176 catalog_manager.cc:6133] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Sending DeleteTablet for 3 replicas of tablet e0648fcd521e4db19359c743447246b7
I20260925 20:00:47.474536 6176 catalog_manager.cc:6934] ReplaceTablet: tablet e0648fcd521e4db19359c743447246b7 deleted and replaced by tablet b06397ec80e1468f9d8b276e6a046b72
I20260925 20:00:47.474936 6174 catalog_manager.cc:6934] ReplaceTablet: tablet 72c3c64108f348a5ab893e6954d72d77 deleted and replaced by tablet a007f75c1b2d4468bc600d75c0a9059d
I20260925 20:00:47.483346 6177 catalog_manager.cc:6133] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Sending DeleteTablet for 3 replicas of tablet b65400989b944ca988d6f26838277336
I20260925 20:00:47.484118 6177 catalog_manager.cc:6934] ReplaceTablet: tablet b65400989b944ca988d6f26838277336 deleted and replaced by tablet 0d9bce361b9542c181d3085db8fcafd3
I20260925 20:00:47.485044 6178 catalog_manager.cc:6133] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Sending DeleteTablet for 3 replicas of tablet 465db2a7c1fd48deb8a09d3366cfc04b
I20260925 20:00:47.485690 6178 catalog_manager.cc:6934] ReplaceTablet: tablet 465db2a7c1fd48deb8a09d3366cfc04b deleted and replaced by tablet d81a8af3280a4d95bf21aa8c7b44890c
@ 0x7f6c548e6ba1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f6c548ee81c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260925 20:00:47.501878 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet 72c3c64108f348a5ab893e6954d72d77 with delete_type TABLET_DATA_DELETED (replaced by tablet a007f75c1b2d4468bc600d75c0a9059d) from {username='slave'} at 127.0.0.1:58354
I20260925 20:00:47.502270 4506 tablet_service.cc:1558] Processing DeleteTablet for tablet e0648fcd521e4db19359c743447246b7 with delete_type TABLET_DATA_DELETED (replaced by tablet b06397ec80e1468f9d8b276e6a046b72) from {username='slave'} at 127.0.0.1:58354
@ 0x7f6c548ee699 std::_Function_handler<>::_M_invoke() at ??:0
I20260925 20:00:47.502476 4504 tablet_service.cc:1558] Processing DeleteTablet for tablet b65400989b944ca988d6f26838277336 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d9bce361b9542c181d3085db8fcafd3) from {username='slave'} at 127.0.0.1:58354
I20260925 20:00:47.503504 6176 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36054:
name: "default.Table_f051ca2de9a64755b465c9b043ce4392"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:47.504679 6178 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36054:
name: "default.Table_30a6c0d501b04bfc882cec2f43c06f83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:47.504915 6177 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36054:
name: "default.Table_9f19364a66794406806691ab3a701b72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:47.504675 6174 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36054:
table { table_name: "default.Table_2ebd7ecb76b44d13a6847408f7fd7d17" } modify_external_catalogs: true
I20260925 20:00:47.505267 6174 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36054:
table { table_name: "default.Table_2ebd7ecb76b44d13a6847408f7fd7d17" } modify_external_catalogs: true
@ 0xaaef9d std::function<>::operator()() at ??:?
I20260925 20:00:47.508955 4503 tablet_service.cc:1558] Processing DeleteTablet for tablet 465db2a7c1fd48deb8a09d3366cfc04b with delete_type TABLET_DATA_DELETED (replaced by tablet d81a8af3280a4d95bf21aa8c7b44890c) from {username='slave'} at 127.0.0.1:58354
I20260925 20:00:47.513005 4644 tablet_service.cc:1558] Processing DeleteTablet for tablet 72c3c64108f348a5ab893e6954d72d77 with delete_type TABLET_DATA_DELETED (replaced by tablet a007f75c1b2d4468bc600d75c0a9059d) from {username='slave'} at 127.0.0.1:36766
I20260925 20:00:47.513258 4646 tablet_service.cc:1558] Processing DeleteTablet for tablet b65400989b944ca988d6f26838277336 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d9bce361b9542c181d3085db8fcafd3) from {username='slave'} at 127.0.0.1:36766
I20260925 20:00:47.513500 4645 tablet_service.cc:1558] Processing DeleteTablet for tablet 465db2a7c1fd48deb8a09d3366cfc04b with delete_type TABLET_DATA_DELETED (replaced by tablet d81a8af3280a4d95bf21aa8c7b44890c) from {username='slave'} at 127.0.0.1:36766
I20260925 20:00:47.513623 4647 tablet_service.cc:1558] Processing DeleteTablet for tablet e0648fcd521e4db19359c743447246b7 with delete_type TABLET_DATA_DELETED (replaced by tablet b06397ec80e1468f9d8b276e6a046b72) from {username='slave'} at 127.0.0.1:36766
@ 0x7f6c40939079 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6c4400f6db start_thread at ??:0
@ 0x7f6c3ca9661f clone at ??:0
I20260925 20:00:47.522131 6270 tablet_replica.cc:333] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:47.522172 6267 tablet_replica.cc:333] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:47.522368 6267 raft_consensus.cc:2243] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:47.522403 6270 raft_consensus.cc:2243] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:47.522755 6270 raft_consensus.cc:2272] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:47.522861 6267 raft_consensus.cc:2272] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:47.523505 6270 ts_tablet_manager.cc:1916] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:47.523574 6267 ts_tablet_manager.cc:1916] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:47.531250 6267 ts_tablet_manager.cc:1929] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:47.531421 6267 log.cc:1199] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/72c3c64108f348a5ab893e6954d72d77
I20260925 20:00:47.531872 6267 ts_tablet_manager.cc:1950] T 72c3c64108f348a5ab893e6954d72d77 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:47.532173 6270 ts_tablet_manager.cc:1929] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:47.532363 6270 log.cc:1199] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/72c3c64108f348a5ab893e6954d72d77
I20260925 20:00:47.532876 6270 ts_tablet_manager.cc:1950] T 72c3c64108f348a5ab893e6954d72d77 P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:47.533087 6267 tablet_replica.cc:333] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:47.533316 6267 raft_consensus.cc:2243] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:47.533519 6267 raft_consensus.cc:2272] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:47.534065 6270 tablet_replica.cc:333] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:47.534125 6267 ts_tablet_manager.cc:1916] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:47.534313 6270 raft_consensus.cc:2243] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:47.534508 6270 raft_consensus.cc:2272] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:47.535151 6270 ts_tablet_manager.cc:1916] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:47.541833 6267 ts_tablet_manager.cc:1929] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:47.541997 6267 log.cc:1199] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/e0648fcd521e4db19359c743447246b7
I20260925 20:00:47.542454 6267 ts_tablet_manager.cc:1950] T e0648fcd521e4db19359c743447246b7 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:47.543263 6270 ts_tablet_manager.cc:1929] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:47.543435 6270 log.cc:1199] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/b65400989b944ca988d6f26838277336
I20260925 20:00:47.543885 6270 ts_tablet_manager.cc:1950] T b65400989b944ca988d6f26838277336 P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:47.543888 6267 tablet_replica.cc:333] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:47.544080 6267 raft_consensus.cc:2243] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:47.544486 6267 raft_consensus.cc:2272] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:47.545095 6267 ts_tablet_manager.cc:1916] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:47.546166 6270 tablet_replica.cc:333] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:47.546413 6270 raft_consensus.cc:2243] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:47.546618 6270 raft_consensus.cc:2272] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:47.548620 6270 ts_tablet_manager.cc:1916] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:47.553102 6267 ts_tablet_manager.cc:1929] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:47.553268 6267 log.cc:1199] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/b65400989b944ca988d6f26838277336
I20260925 20:00:47.553758 6267 ts_tablet_manager.cc:1950] T b65400989b944ca988d6f26838277336 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:47.554935 6267 tablet_replica.cc:333] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:47.555171 6267 raft_consensus.cc:2243] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:47.555385 6267 raft_consensus.cc:2272] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:47.556137 6267 ts_tablet_manager.cc:1916] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:47.556353 6270 ts_tablet_manager.cc:1929] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:47.556599 6270 log.cc:1199] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/465db2a7c1fd48deb8a09d3366cfc04b
I20260925 20:00:47.557097 6270 ts_tablet_manager.cc:1950] T 465db2a7c1fd48deb8a09d3366cfc04b P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:47.558362 6270 tablet_replica.cc:333] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:47.558570 6270 raft_consensus.cc:2243] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:47.558755 6270 raft_consensus.cc:2272] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:47.559345 6270 ts_tablet_manager.cc:1916] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:47.564031 6267 ts_tablet_manager.cc:1929] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:47.564208 6267 log.cc:1199] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/465db2a7c1fd48deb8a09d3366cfc04b
I20260925 20:00:47.564747 6267 ts_tablet_manager.cc:1950] T 465db2a7c1fd48deb8a09d3366cfc04b P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:47.567283 6270 ts_tablet_manager.cc:1929] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:47.567437 6270 log.cc:1199] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/e0648fcd521e4db19359c743447246b7
I20260925 20:00:47.567848 6270 ts_tablet_manager.cc:1950] T e0648fcd521e4db19359c743447246b7 P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
W20260925 20:00:47.649039 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 1: this message will repeat every 5th retry.
W20260925 20:00:47.727248 4600 connection.cc:573] server connection from 127.0.0.1:36766 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:47.727375 4455 connection.cc:573] server connection from 127.0.0.1:58354 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:47.727494 4862 connection.cc:573] client connection to 127.3.199.253:41897 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:47.727810 6262 negotiation.cc:336] Failed RPC negotiation. Trace:
0925 20:00:47.473609 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:39520 (local address 127.3.199.195:34395)
0925 20:00:47.496839 (+ 23230us) server_negotiation.cc:207] Beginning negotiation
0925 20:00:47.496849 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0925 20:00:47.496873 (+ 24us) server_negotiation.cc:408] Connection header received
0925 20:00:47.496955 (+ 82us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0925 20:00:47.496963 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
0925 20:00:47.497067 (+ 104us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0925 20:00:47.497210 (+ 143us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0925 20:00:47.519332 (+ 22122us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0925 20:00:47.521477 (+ 2145us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0925 20:00:47.727517 (+206040us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39520: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":23067,"thread_start_us":23860,"threads_started":1}
W20260925 20:00:47.728178 4740 connection.cc:573] client connection to 127.3.199.253:41897 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:47.729031 4862 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.3.199.253:41897)
W20260925 20:00:47.729727 4862 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.199.253:41897)
W20260925 20:00:47.729959 4862 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.199.253:41897)
W20260925 20:00:47.730186 4862 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.3.199.253:41897)
W20260925 20:00:47.810949 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 1: this message will repeat every 5th retry.
I20260925 20:00:48.012753 6281 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:48.013046 6281 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:48.013833 6281 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:48.016692 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:48.016952 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:48.017136 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:48.017529 6281 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260925 20:00:48.018371 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 1: this message will repeat every 5th retry.
I20260925 20:00:48.064802 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:00:48.065047 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:00:48.360401 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 1: this message will repeat every 5th retry.
I20260925 20:00:48.689244 6287 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:48.689532 6287 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:48.690346 6287 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:48.691303 6031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260925 20:00:48.692658 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:48.693164 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:48.693320 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:48.693662 6287 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260925 20:00:49.445410 6296 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:49.445662 6296 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:49.446384 6296 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:49.448860 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:49.449723 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:49.449870 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:49.450237 6296 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260925 20:00:49.924003 3871 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
/tmp/dist-test-taskg5SBXY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.253:41897
--webserver_interface=127.3.199.253
--webserver_port=41541
--builtin_ntp_servers=127.3.199.212:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 20:00:49.932610 6292 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:00:49.932858 6292 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:49.933900 6292 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:00:49.934943 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:00:49.935170 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:00:49.935341 4741 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:00:49.935669 6292 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:00:50.085768 6303 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:00:50.086544 6303 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
W20260925 20:00:50.089483 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:00:50.089969 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:00:50.090147 4739 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:00:50.090595 6292 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:00:50.092147 6303 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:50.295745 6306 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:50.296109 6306 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:50.297098 6306 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:50.301316 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:50.301633 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:50.301772 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:50.302063 6306 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260925 20:00:50.326668 6305 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 20:00:50.327013 6305 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 20:00:50.327085 6305 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 20:00:50.336638 6305 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 20:00:50.336750 6305 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 20:00:50.336807 6305 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 20:00:50.336855 6305 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 20:00:50.348372 6305 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.199.212:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:37275
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.199.253:41897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.199.253
--webserver_port=41541
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 25 Sep 2026 19:43:21 UTC on bdcb31816ec0
build id 13997
ASAN enabled
I20260925 20:00:50.350025 6305 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 20:00:50.351850 6305 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 20:00:50.360985 6315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:50.361032 6317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 20:00:50.360981 6314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 20:00:50.362581 6305 server_base.cc:1061] running on GCE node
I20260925 20:00:50.363554 6305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 20:00:50.365295 6305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 20:00:50.366577 6305 hybrid_clock.cc:648] HybridClock initialized: now 1790366450366532 us; error 43 us; skew 500 ppm
I20260925 20:00:50.367026 6305 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 20:00:50.369625 6305 webserver.cc:533] Webserver started at http://127.3.199.253:41541/ using document root <none> and password file <none>
I20260925 20:00:50.370774 6305 fs_manager.cc:362] Metadata directory not provided
I20260925 20:00:50.371096 6305 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 20:00:50.377779 6305 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 20:00:50.381279 6323 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 20:00:50.382877 6305 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 20:00:50.383139 6305 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data,/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
uuid: "da29ee550f82456fa7c5ce7ea916909f"
format_stamp: "Formatted at 2026-09-25 20:00:17 on dist-test-slave-9gwk"
I20260925 20:00:50.383986 6305 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 20:00:50.411158 6305 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 20:00:50.412127 6305 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 20:00:50.412513 6305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 20:00:50.436225 6374 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.199.253:41897 every 8 connection(s)
I20260925 20:00:50.436257 6305 rpc_server.cc:307] RPC server started. Bound to: 127.3.199.253:41897
I20260925 20:00:50.438257 6305 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/master-1/data/info.pb
I20260925 20:00:50.442123 3871 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskg5SBXY/build/asan/bin/kudu as pid 6305
I20260925 20:00:50.448951 6375 sys_catalog.cc:263] Verifying existing consensus state
I20260925 20:00:50.456341 6375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Bootstrap starting.
I20260925 20:00:50.477488 6375 log.cc:826] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Log is configured to *not* fsync() on all Append() calls
I20260925 20:00:50.561655 6375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Bootstrap replayed 1/1 log segments. Stats: ops{read=105 overwritten=0 applied=104 ignored=0} inserts{seen=62 ignored=0} mutations{seen=85 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260925 20:00:50.562515 6375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Bootstrap complete.
I20260925 20:00:50.569540 6375 raft_consensus.cc:359] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 9 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:50.572597 6375 raft_consensus.cc:740] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: da29ee550f82456fa7c5ce7ea916909f, State: Initialized, Role: FOLLOWER
I20260925 20:00:50.573709 6375 consensus_queue.cc:260] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 104, Last appended: 9.105, Last appended by leader: 105, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:50.578051 6382 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 9 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:50.578562 6382 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:50.579654 6375 sys_catalog.cc:565] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 20:00:50.602490 6396 catalog_manager.cc:1308] Loaded cluster ID: 7efb83138ca94380bf2d8f0f2f91c381
I20260925 20:00:50.602926 6396 catalog_manager.cc:1601] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: loading cluster ID for follower catalog manager: success
I20260925 20:00:50.607630 6396 catalog_manager.cc:1623] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: acquiring CA information for follower catalog manager: success
I20260925 20:00:50.613072 6396 catalog_manager.cc:1651] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 20:00:50.633366 6305 master_runner.cc:187] Error getting master registration for 127.3.199.252:37957: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111),
I20260925 20:00:50.639966 6305 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:00:50.778976 4852 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:50.782606 6339 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" instance_seqno: 1790366420065491) as {username='slave'} at 127.3.199.195:57243; Asking this server to re-register.
I20260925 20:00:50.783612 4852 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:50.786065 6339 ts_manager.cc:194] Registered new tserver with Master: dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:50.817889 4573 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:50.819867 6339 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" instance_seqno: 1790366418849966) as {username='slave'} at 127.3.199.193:49535; Asking this server to re-register.
I20260925 20:00:50.820542 4714 heartbeater.cc:344] Connected to a master server at 127.3.199.253:41897
I20260925 20:00:50.820866 4573 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:50.822419 6339 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" instance_seqno: 1790366419525613) as {username='slave'} at 127.3.199.194:32805; Asking this server to re-register.
I20260925 20:00:50.823089 6340 ts_manager.cc:194] Registered new tserver with Master: 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741)
I20260925 20:00:50.823863 4714 heartbeater.cc:461] Registering TS with master...
I20260925 20:00:50.825563 6340 ts_manager.cc:194] Registered new tserver with Master: 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
I20260925 20:00:51.170831 6407 raft_consensus.cc:493] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:51.171133 6407 raft_consensus.cc:515] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:51.173077 6407 leader_election.cc:290] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:51.173722 6055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da29ee550f82456fa7c5ce7ea916909f" candidate_term: 10 candidate_status { last_received { term: 9 index: 105 } } ignore_live_leader: false dest_uuid: "169b6de20560440b99cc08af549ce477" is_pre_election: true
I20260925 20:00:51.174136 6055 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da29ee550f82456fa7c5ce7ea916909f in term 9.
W20260925 20:00:51.174305 6327 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:51.174887 6326 leader_election.cc:304] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 169b6de20560440b99cc08af549ce477, da29ee550f82456fa7c5ce7ea916909f; no voters:
I20260925 20:00:51.175410 6407 raft_consensus.cc:2804] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260925 20:00:51.175586 6407 raft_consensus.cc:493] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:51.175707 6407 raft_consensus.cc:3060] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 9 FOLLOWER]: Advancing to term 10
W20260925 20:00:51.176393 6327 leader_election.cc:336] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:51.179409 6407 raft_consensus.cc:515] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:51.180055 6407 leader_election.cc:290] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 10 election: Requested vote from peers 169b6de20560440b99cc08af549ce477 (127.3.199.254:41289), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
I20260925 20:00:51.180794 6055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da29ee550f82456fa7c5ce7ea916909f" candidate_term: 10 candidate_status { last_received { term: 9 index: 105 } } ignore_live_leader: false dest_uuid: "169b6de20560440b99cc08af549ce477"
I20260925 20:00:51.181036 6055 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 9 FOLLOWER]: Advancing to term 10
W20260925 20:00:51.182324 6327 leader_election.cc:336] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:51.183580 6055 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da29ee550f82456fa7c5ce7ea916909f in term 10.
I20260925 20:00:51.184132 6326 leader_election.cc:304] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 169b6de20560440b99cc08af549ce477, da29ee550f82456fa7c5ce7ea916909f; no voters: 7f43a6a14ef547f2aeb22ce2b334383e
I20260925 20:00:51.184556 6407 raft_consensus.cc:2804] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 10 FOLLOWER]: Leader election won for term 10
I20260925 20:00:51.184991 6407 raft_consensus.cc:697] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [term 10 LEADER]: Becoming Leader. State: Replica: da29ee550f82456fa7c5ce7ea916909f, State: Running, Role: LEADER
I20260925 20:00:51.185647 6407 consensus_queue.cc:237] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 104, Committed index: 104, Last appended: 9.105, Last appended by leader: 105, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:51.188740 6410 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: New leader da29ee550f82456fa7c5ce7ea916909f. Latest consensus state: current_term: 10 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:51.189095 6410 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: LEADER
W20260925 20:00:51.579213 6327 consensus_peers.cc:597] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f -> Peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Couldn't send request to peer 7f43a6a14ef547f2aeb22ce2b334383e. Status: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 20:00:51.656158 6055 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Refusing update from remote peer da29ee550f82456fa7c5ce7ea916909f: Log matching property violated. Preceding OpId in replica: term: 9 index: 105. Preceding OpId from leader: term: 10 index: 106. (index mismatch)
I20260925 20:00:51.657341 6410 consensus_queue.cc:1048] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [LEADER]: Connected to new peer: Peer: permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 106, Last known committed idx: 104, Time since last communication: 0.000s
I20260925 20:00:51.659984 6414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da29ee550f82456fa7c5ce7ea916909f. Latest consensus state: current_term: 10 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:51.660399 6414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:51.664623 6410 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 10 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:51.665000 6410 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f [sys.catalog]: This master's current role is: LEADER
I20260925 20:00:51.666985 6414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 10 leader_uuid: "da29ee550f82456fa7c5ce7ea916909f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } } }
I20260925 20:00:51.667276 6414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 20:00:51.707373 6412 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260925 20:00:51.710147 6412 catalog_manager.cc:692] Loaded metadata for table default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330]
I20260925 20:00:51.710592 6412 catalog_manager.cc:692] Loaded metadata for table default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf]
I20260925 20:00:51.710912 6412 catalog_manager.cc:692] Loaded metadata for table default.table_703616ef44ba43f9afb9c4df73eb372c [id=25f6e1ade43c4519bed519037ee8f761]
I20260925 20:00:51.711193 6412 catalog_manager.cc:692] Loaded metadata for table default.table_667eace0e712438bbd1211f7e3e665f8 [id=439b3d65790141d1abfbca4788836f80]
I20260925 20:00:51.711465 6412 catalog_manager.cc:692] Loaded metadata for table default.table_127c3d62fd1c4394bcd55bc6fc18bc5d [id=45cbddfbaced49a989c1fb61a644a713]
I20260925 20:00:51.711767 6412 catalog_manager.cc:692] Loaded metadata for table default.table_2ebd7ecb76b44d13a6847408f7fd7d17 [id=4b80e3afa6954e2ea8b6d1b5ec79a9d1]
I20260925 20:00:51.712044 6412 catalog_manager.cc:692] Loaded metadata for table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812]
I20260925 20:00:51.712525 6412 catalog_manager.cc:692] Loaded metadata for table default.table_c78f9cc1d7e74b51b29858c7d3248fb8 [id=7ead82097c49448f85b1dbef027be066]
I20260925 20:00:51.712917 6412 catalog_manager.cc:692] Loaded metadata for table default.table_098b200909f842c88a38f11192f4dc25 [id=86f1c8a16cc14ef9b72c95060313b3ad]
I20260925 20:00:51.713308 6412 catalog_manager.cc:692] Loaded metadata for table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22]
I20260925 20:00:51.713712 6412 catalog_manager.cc:692] Loaded metadata for table default.table_5b8386185e8e4fcd99110bd5137a578c [id=a385e0b7db6a49e8b1baf34a5fdd3473]
I20260925 20:00:51.714557 6412 catalog_manager.cc:692] Loaded metadata for table default.table_7722a249d10f4616ba5b5bb2fc149c18 [id=a8f568d7faab469ba7ea8a2a38bfd638]
I20260925 20:00:51.714969 6412 catalog_manager.cc:692] Loaded metadata for table default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d]
I20260925 20:00:51.715310 6412 catalog_manager.cc:692] Loaded metadata for table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7]
I20260925 20:00:51.715798 6412 catalog_manager.cc:692] Loaded metadata for table default.table_08807903f9884f2a9707d4066e7f2f52 [id=f6fa165ad5e7471589cb63403548ab2c]
I20260925 20:00:51.718199 6412 tablet_loader.cc:96] loaded metadata for tablet 0d9bce361b9542c181d3085db8fcafd3 (table default.table_667eace0e712438bbd1211f7e3e665f8 [id=439b3d65790141d1abfbca4788836f80])
I20260925 20:00:51.718688 6412 tablet_loader.cc:96] loaded metadata for tablet 2ddb9fbd0aeb4c4a985508ac67c93227 (table default.table_7722a249d10f4616ba5b5bb2fc149c18 [id=a8f568d7faab469ba7ea8a2a38bfd638])
I20260925 20:00:51.719012 6412 tablet_loader.cc:96] loaded metadata for tablet 44f42691903d4e2bb3f24ecb75a3f86e (table default.table_08807903f9884f2a9707d4066e7f2f52 [id=f6fa165ad5e7471589cb63403548ab2c])
I20260925 20:00:51.719602 6412 tablet_loader.cc:96] loaded metadata for tablet 5982cfa34e4e40d3894cbf46aab33715 (table default.table_2ebd7ecb76b44d13a6847408f7fd7d17 [id=4b80e3afa6954e2ea8b6d1b5ec79a9d1])
I20260925 20:00:51.719935 6412 tablet_loader.cc:96] loaded metadata for tablet 59a14370409146a1847c5b76755e3feb (table default.table_c78f9cc1d7e74b51b29858c7d3248fb8 [id=7ead82097c49448f85b1dbef027be066])
I20260925 20:00:51.720580 6412 tablet_loader.cc:96] loaded metadata for tablet 653a4121e8934471b99a9853b73617ab (table default.table_5b8386185e8e4fcd99110bd5137a578c [id=a385e0b7db6a49e8b1baf34a5fdd3473])
I20260925 20:00:51.721016 6412 tablet_loader.cc:96] loaded metadata for tablet 81d09be24f0a4687a1b17ddd02fd4e47 (table default.table_b74d63554f344195800ce065c2a3cddf [id=90221d8ab3b94046bbdff933844c7f22])
I20260925 20:00:51.721306 6412 tablet_loader.cc:96] loaded metadata for tablet 83c0e7bd6ef040b9af169d9677137836 (table default.table_b6819354070b414c8a47d9f19a011a5c [id=189081df4c034817b16de973c311b330])
I20260925 20:00:51.722150 6412 tablet_loader.cc:96] loaded metadata for tablet a007f75c1b2d4468bc600d75c0a9059d (table default.table_e72a162b7a554904a9ca0cceb5e6b273 [id=78f973c6a62948e18d7f86d475b64812])
I20260925 20:00:51.722522 6412 tablet_loader.cc:96] loaded metadata for tablet b06397ec80e1468f9d8b276e6a046b72 (table default.table_703616ef44ba43f9afb9c4df73eb372c [id=25f6e1ade43c4519bed519037ee8f761])
I20260925 20:00:51.722847 6412 tablet_loader.cc:96] loaded metadata for tablet b321f36472cc42babf1ec3f64ab990ed (table default.table_c860d14624744708bf4ff50a84517b59 [id=c3a1c6d72e6b46c4b253c1c356b17bd7])
I20260925 20:00:51.723161 6412 tablet_loader.cc:96] loaded metadata for tablet b59f74f459c542a88e8bef5f1c51357d (table default.table_5fdc6db4e1ee47848136cec254f09a72 [id=2411a96fbe974a87a2ef047732f0b4bf])
I20260925 20:00:51.723666 6412 tablet_loader.cc:96] loaded metadata for tablet d56227e2824d41beb764a79713226482 (table default.table_127c3d62fd1c4394bcd55bc6fc18bc5d [id=45cbddfbaced49a989c1fb61a644a713])
I20260925 20:00:51.723968 6412 tablet_loader.cc:96] loaded metadata for tablet d81a8af3280a4d95bf21aa8c7b44890c (table default.table_cf7560049cf84f22b27dd94a6b4dbcef [id=b25b053e66e341d4bcb5a1c4dee4983d])
I20260925 20:00:51.725057 6412 tablet_loader.cc:96] loaded metadata for tablet ee843d55757347ef9942dffd917c0ff8 (table default.table_098b200909f842c88a38f11192f4dc25 [id=86f1c8a16cc14ef9b72c95060313b3ad])
I20260925 20:00:51.725729 6412 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260925 20:00:51.726888 6412 catalog_manager.cc:1308] Loaded cluster ID: 7efb83138ca94380bf2d8f0f2f91c381
I20260925 20:00:51.726989 6412 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260925 20:00:51.728515 6412 catalog_manager.cc:1553] Loading token signing keys...
I20260925 20:00:51.729612 6412 catalog_manager.cc:6230] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Loaded TSK: 0
I20260925 20:00:51.730938 6412 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260925 20:00:51.731998 6412 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260925 20:00:51.733071 6412 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: 20
I20260925 20:00:51.768359 6340 master_service.cc:960] ReplaceTablet: received request to replace tablet 83c0e7bd6ef040b9af169d9677137836 from {username='slave'} at 127.0.0.1:36708
I20260925 20:00:51.768676 6338 master_service.cc:960] ReplaceTablet: received request to replace tablet b321f36472cc42babf1ec3f64ab990ed from {username='slave'} at 127.0.0.1:36708
I20260925 20:00:51.768738 6337 master_service.cc:960] ReplaceTablet: received request to replace tablet 44f42691903d4e2bb3f24ecb75a3f86e from {username='slave'} at 127.0.0.1:36708
I20260925 20:00:51.768404 6339 master_service.cc:960] ReplaceTablet: received request to replace tablet b59f74f459c542a88e8bef5f1c51357d from {username='slave'} at 127.0.0.1:36708
I20260925 20:00:51.769248 6336 master_service.cc:960] ReplaceTablet: received request to replace tablet 81d09be24f0a4687a1b17ddd02fd4e47 from {username='slave'} at 127.0.0.1:36708
I20260925 20:00:51.770196 6335 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36708:
table { table_name: "default.Table_e72a162b7a554904a9ca0cceb5e6b273" } new_table_name: "default.Table_1073b55979e5429c83265f7d64835607" modify_external_catalogs: true
I20260925 20:00:51.773663 6334 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36708:
name: "default.Table_799678007be3498da115944e990a1785"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:51.775069 6333 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36708:
table { table_name: "default.Table_cf7560049cf84f22b27dd94a6b4dbcef" } modify_external_catalogs: true
I20260925 20:00:51.775355 6333 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36708:
table { table_name: "default.Table_cf7560049cf84f22b27dd94a6b4dbcef" } modify_external_catalogs: true
I20260925 20:00:51.776176 6332 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36708:
table { table_name: "default.Table_c78f9cc1d7e74b51b29858c7d3248fb8" } new_table_name: "default.Table_1f01c68bbe404907a4d5f4d12fb9ee26" modify_external_catalogs: true
I20260925 20:00:51.776384 6331 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36708:
name: "default.Table_ef69b46f2522492baaac95731da2bbcb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260925 20:00:51.780927 6395 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 21, previous 21; dumping first 2 out of 13 received events: NotificationEvent(eventId=21, eventTime=1790366425, eventType=DROP_TABLE, dbName=default, tableName=table_30a6c0d501b04bfc882cec2f43c06f83, message={"server":"","servicePrincipal":"","db":"default","table":"table_30a6c0d501b04bfc882cec2f43c06f83","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_30a6c0d501b04bfc882cec2f43c06f83\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790366424},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/warehouse/table_30a6c0d501b04bfc882cec2f43c06f83\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_30a6c0d501b04bfc882cec2f43c06f83\",\"kudu.table_id\":\"44287e882a4040dcb79d824c08808dd4\",\"transient_lastDdlTime\":\"1790366424\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"7efb83138ca94380bf2d8f0f2f91c381\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790366425}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=21, eventTime=1790366425, eventType=CREATE_TABLE, dbName=default, tableName=table_127c3d62fd1c4394bcd55bc6fc18bc5d, message={"server":"","servicePrincipal":"","db":"default","table":"table_127c3d62fd1c4394bcd55bc6fc18bc5d","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_127c3d62fd1c4394bcd55bc6fc18bc5d\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790366425},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/warehouse/table_127c3d62fd1c4394bcd55bc6fc18bc5d\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_127c3d62fd1c4394bcd55bc6fc18bc5d\",\"kudu.table_id\":\"45cbddfbaced49a989c1fb61a644a713\",\"transient_lastDdlTime\":\"1790366425\",\"comment\":\"\",\"kudu.master_addresses\":\"127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"7efb83138ca94380bf2d8f0f2f91c381\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790366425,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7fac8f4f2d5d google::LogMessage::Fail() at ??:0
@ 0x7fac8f4f6b23 google::LogMessage::SendToLog() at ??:0
@ 0x7fac8f4f275c google::LogMessage::Flush() at ??:0
@ 0x7fac8f4f29a1 google::LogMessage::~LogMessage() at ??:0
I20260925 20:00:51.791563 6337 catalog_manager.cc:6133] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Sending DeleteTablet for 3 replicas of tablet 44f42691903d4e2bb3f24ecb75a3f86e
@ 0x7faca61f12f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260925 20:00:51.802896 6338 catalog_manager.cc:6133] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Sending DeleteTablet for 3 replicas of tablet b321f36472cc42babf1ec3f64ab990ed
I20260925 20:00:51.804466 6338 catalog_manager.cc:6934] ReplaceTablet: tablet b321f36472cc42babf1ec3f64ab990ed deleted and replaced by tablet fc4d1ba798ea4a5a94e2e307f53269eb
I20260925 20:00:51.805262 6337 catalog_manager.cc:6934] ReplaceTablet: tablet 44f42691903d4e2bb3f24ecb75a3f86e deleted and replaced by tablet fd68c5c05f5948b981de64b6785da41a
I20260925 20:00:51.808790 6336 catalog_manager.cc:6133] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Sending DeleteTablet for 3 replicas of tablet 81d09be24f0a4687a1b17ddd02fd4e47
@ 0x7faca61efba1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260925 20:00:51.809522 6336 catalog_manager.cc:6934] ReplaceTablet: tablet 81d09be24f0a4687a1b17ddd02fd4e47 deleted and replaced by tablet f1193f791d6f45d78ac4f49d54f5c38e
@ 0x7faca61f781c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7faca61f7699 std::_Function_handler<>::_M_invoke() at ??:0
I20260925 20:00:51.831504 6340 catalog_manager.cc:6133] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Sending DeleteTablet for 3 replicas of tablet 83c0e7bd6ef040b9af169d9677137836
I20260925 20:00:51.832268 6340 catalog_manager.cc:6934] ReplaceTablet: tablet 83c0e7bd6ef040b9af169d9677137836 deleted and replaced by tablet 22de021e70984e5e88b3c722b1284820
I20260925 20:00:51.835122 6336 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36708:
table { table_name: "default.Table_127c3d62fd1c4394bcd55bc6fc18bc5d" } modify_external_catalogs: true
I20260925 20:00:51.835517 6336 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36708:
table { table_name: "default.Table_127c3d62fd1c4394bcd55bc6fc18bc5d" } modify_external_catalogs: true
I20260925 20:00:51.836165 6337 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36708:
name: "default.Table_d4b20390d75f45c89df2fe5d8498b337"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:51.838441 6338 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36708:
name: "default.Table_4627e927cd124d049b0fdc22eb15c554"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 20:00:51.838549 4785 tablet_service.cc:1558] Processing DeleteTablet for tablet 44f42691903d4e2bb3f24ecb75a3f86e with delete_type TABLET_DATA_DELETED (replaced by tablet fd68c5c05f5948b981de64b6785da41a) from {username='slave'} at 127.0.0.1:39536
I20260925 20:00:51.838943 4784 tablet_service.cc:1558] Processing DeleteTablet for tablet 81d09be24f0a4687a1b17ddd02fd4e47 with delete_type TABLET_DATA_DELETED (replaced by tablet f1193f791d6f45d78ac4f49d54f5c38e) from {username='slave'} at 127.0.0.1:39536
I20260925 20:00:51.839037 4782 tablet_service.cc:1558] Processing DeleteTablet for tablet b321f36472cc42babf1ec3f64ab990ed with delete_type TABLET_DATA_DELETED (replaced by tablet fc4d1ba798ea4a5a94e2e307f53269eb) from {username='slave'} at 127.0.0.1:39536
I20260925 20:00:51.839424 4780 tablet_service.cc:1558] Processing DeleteTablet for tablet 83c0e7bd6ef040b9af169d9677137836 with delete_type TABLET_DATA_DELETED (replaced by tablet 22de021e70984e5e88b3c722b1284820) from {username='slave'} at 127.0.0.1:39536
I20260925 20:00:51.839609 4852 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:51.840276 6340 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36708:
name: "default.Table_aae983110c484d68a3a085faa1688c5e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0xaaef9d std::function<>::operator()() at ??:?
I20260925 20:00:51.847009 6339 catalog_manager.cc:6133] T 00000000000000000000000000000000 P da29ee550f82456fa7c5ce7ea916909f: Sending DeleteTablet for 3 replicas of tablet b59f74f459c542a88e8bef5f1c51357d
I20260925 20:00:51.847895 6339 catalog_manager.cc:6934] ReplaceTablet: tablet b59f74f459c542a88e8bef5f1c51357d deleted and replaced by tablet 27e00dcaeaa342c79a2b4588382ffd6a
@ 0x7fac92242079 kudu::Thread::SuperviseThread() at ??:0
I20260925 20:00:51.852670 6437 tablet_replica.cc:333] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:51.853206 4780 tablet_service.cc:1558] Processing DeleteTablet for tablet b59f74f459c542a88e8bef5f1c51357d with delete_type TABLET_DATA_DELETED (replaced by tablet 27e00dcaeaa342c79a2b4588382ffd6a) from {username='slave'} at 127.0.0.1:39536
I20260925 20:00:51.853502 6437 raft_consensus.cc:2243] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:51.853746 6437 raft_consensus.cc:2272] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:51.854394 6437 ts_tablet_manager.cc:1916] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7fac959186db start_thread at ??:0
@ 0x7fac8e39f61f clone at ??:0
I20260925 20:00:51.859594 4505 tablet_service.cc:1558] Processing DeleteTablet for tablet b321f36472cc42babf1ec3f64ab990ed with delete_type TABLET_DATA_DELETED (replaced by tablet fc4d1ba798ea4a5a94e2e307f53269eb) from {username='slave'} at 127.0.0.1:58370
I20260925 20:00:51.859715 4644 tablet_service.cc:1558] Processing DeleteTablet for tablet 44f42691903d4e2bb3f24ecb75a3f86e with delete_type TABLET_DATA_DELETED (replaced by tablet fd68c5c05f5948b981de64b6785da41a) from {username='slave'} at 127.0.0.1:36778
I20260925 20:00:51.859999 4503 tablet_service.cc:1558] Processing DeleteTablet for tablet 44f42691903d4e2bb3f24ecb75a3f86e with delete_type TABLET_DATA_DELETED (replaced by tablet fd68c5c05f5948b981de64b6785da41a) from {username='slave'} at 127.0.0.1:58370
I20260925 20:00:51.860299 4504 tablet_service.cc:1558] Processing DeleteTablet for tablet 81d09be24f0a4687a1b17ddd02fd4e47 with delete_type TABLET_DATA_DELETED (replaced by tablet f1193f791d6f45d78ac4f49d54f5c38e) from {username='slave'} at 127.0.0.1:58370
I20260925 20:00:51.861183 4501 tablet_service.cc:1511] Processing CreateTablet for tablet fd68c5c05f5948b981de64b6785da41a (DEFAULT_TABLE table=default.table_08807903f9884f2a9707d4066e7f2f52 [id=f6fa165ad5e7471589cb63403548ab2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:51.862093 4501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd68c5c05f5948b981de64b6785da41a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:51.862810 4647 tablet_service.cc:1558] Processing DeleteTablet for tablet b321f36472cc42babf1ec3f64ab990ed with delete_type TABLET_DATA_DELETED (replaced by tablet fc4d1ba798ea4a5a94e2e307f53269eb) from {username='slave'} at 127.0.0.1:36778
I20260925 20:00:51.863667 4647 tablet_service.cc:1558] Processing DeleteTablet for tablet 81d09be24f0a4687a1b17ddd02fd4e47 with delete_type TABLET_DATA_DELETED (replaced by tablet f1193f791d6f45d78ac4f49d54f5c38e) from {username='slave'} at 127.0.0.1:36778
I20260925 20:00:51.864271 4647 tablet_service.cc:1558] Processing DeleteTablet for tablet 83c0e7bd6ef040b9af169d9677137836 with delete_type TABLET_DATA_DELETED (replaced by tablet 22de021e70984e5e88b3c722b1284820) from {username='slave'} at 127.0.0.1:36778
I20260925 20:00:51.864562 4502 tablet_service.cc:1558] Processing DeleteTablet for tablet b59f74f459c542a88e8bef5f1c51357d with delete_type TABLET_DATA_DELETED (replaced by tablet 27e00dcaeaa342c79a2b4588382ffd6a) from {username='slave'} at 127.0.0.1:58370
I20260925 20:00:51.864825 4647 tablet_service.cc:1558] Processing DeleteTablet for tablet b59f74f459c542a88e8bef5f1c51357d with delete_type TABLET_DATA_DELETED (replaced by tablet 27e00dcaeaa342c79a2b4588382ffd6a) from {username='slave'} at 127.0.0.1:36778
I20260925 20:00:51.864935 4506 tablet_service.cc:1558] Processing DeleteTablet for tablet 83c0e7bd6ef040b9af169d9677137836 with delete_type TABLET_DATA_DELETED (replaced by tablet 22de021e70984e5e88b3c722b1284820) from {username='slave'} at 127.0.0.1:58370
I20260925 20:00:51.865676 4647 tablet_service.cc:1511] Processing CreateTablet for tablet fd68c5c05f5948b981de64b6785da41a (DEFAULT_TABLE table=default.table_08807903f9884f2a9707d4066e7f2f52 [id=f6fa165ad5e7471589cb63403548ab2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 20:00:51.866253 4647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd68c5c05f5948b981de64b6785da41a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:51.878860 6439 tablet_replica.cc:333] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:51.879142 6439 raft_consensus.cc:2243] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:51.879647 6439 raft_consensus.cc:2272] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:51.880445 6439 ts_tablet_manager.cc:1916] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:51.885752 6437 ts_tablet_manager.cc:1929] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:51.885949 6437 log.cc:1199] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/44f42691903d4e2bb3f24ecb75a3f86e
I20260925 20:00:51.886502 6437 ts_tablet_manager.cc:1950] T 44f42691903d4e2bb3f24ecb75a3f86e P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:51.888710 6439 ts_tablet_manager.cc:1929] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:51.888859 6439 log.cc:1199] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/b321f36472cc42babf1ec3f64ab990ed
I20260925 20:00:51.889565 6439 ts_tablet_manager.cc:1950] T b321f36472cc42babf1ec3f64ab990ed P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:51.890178 6440 tablet_replica.cc:333] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:51.890406 6440 raft_consensus.cc:2243] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:51.890839 6440 raft_consensus.cc:2272] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:51.891448 6439 tablet_replica.cc:333] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:51.891474 6440 ts_tablet_manager.cc:1916] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:51.891681 6439 raft_consensus.cc:2243] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:51.891889 6439 raft_consensus.cc:2272] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:51.892760 6437 tablet_replica.cc:333] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:51.892697 6439 ts_tablet_manager.cc:1916] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:51.892997 6437 raft_consensus.cc:2243] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:51.895031 6441 tablet_bootstrap.cc:492] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0: Bootstrap starting.
I20260925 20:00:51.897419 6441 tablet_bootstrap.cc:654] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:51.900007 6442 tablet_bootstrap.cc:492] T fd68c5c05f5948b981de64b6785da41a P 145cdcf7953e4c0d9686b1c6a86f9482: Bootstrap starting.
I20260925 20:00:51.901029 6439 ts_tablet_manager.cc:1929] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:51.901193 6439 log.cc:1199] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/44f42691903d4e2bb3f24ecb75a3f86e
I20260925 20:00:51.901722 6439 ts_tablet_manager.cc:1950] T 44f42691903d4e2bb3f24ecb75a3f86e P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:51.902925 6442 tablet_bootstrap.cc:654] T fd68c5c05f5948b981de64b6785da41a P 145cdcf7953e4c0d9686b1c6a86f9482: Neither blocks nor log segments found. Creating new log.
I20260925 20:00:51.902973 6439 tablet_replica.cc:333] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:51.903178 6439 raft_consensus.cc:2243] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:51.903383 6439 raft_consensus.cc:2272] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:51.904801 6437 raft_consensus.cc:2272] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:51.904850 6439 ts_tablet_manager.cc:1916] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:51.905508 6437 ts_tablet_manager.cc:1916] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:51.909070 6441 tablet_bootstrap.cc:492] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0: No bootstrap required, opened a new log
I20260925 20:00:51.909256 6441 ts_tablet_manager.cc:1403] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.004s
I20260925 20:00:51.910185 6441 raft_consensus.cc:359] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:51.910413 6441 raft_consensus.cc:385] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:51.910514 6441 raft_consensus.cc:740] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Initialized, Role: FOLLOWER
I20260925 20:00:51.914331 6440 ts_tablet_manager.cc:1929] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:51.914458 6440 log.cc:1199] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/44f42691903d4e2bb3f24ecb75a3f86e
I20260925 20:00:51.914897 6440 ts_tablet_manager.cc:1950] T 44f42691903d4e2bb3f24ecb75a3f86e P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:51.916196 6437 ts_tablet_manager.cc:1929] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:51.916361 6437 log.cc:1199] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/81d09be24f0a4687a1b17ddd02fd4e47
I20260925 20:00:51.916805 6437 ts_tablet_manager.cc:1950] T 81d09be24f0a4687a1b17ddd02fd4e47 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:51.917801 6441 consensus_queue.cc:260] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:51.919227 4714 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:51.919821 6439 ts_tablet_manager.cc:1929] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:51.919936 6439 log.cc:1199] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/81d09be24f0a4687a1b17ddd02fd4e47
I20260925 20:00:51.920261 6440 tablet_replica.cc:333] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:51.920361 6439 ts_tablet_manager.cc:1950] T 81d09be24f0a4687a1b17ddd02fd4e47 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:51.920580 6440 raft_consensus.cc:2243] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:51.920807 6440 raft_consensus.cc:2272] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:51.921423 6440 ts_tablet_manager.cc:1916] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:51.923091 6441 ts_tablet_manager.cc:1434] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0: Time spent starting tablet: real 0.014s user 0.000s sys 0.003s
I20260925 20:00:51.923573 6439 tablet_replica.cc:333] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:51.923791 6439 raft_consensus.cc:2243] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:51.923975 6439 raft_consensus.cc:2272] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:51.924672 6439 ts_tablet_manager.cc:1916] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:51.926077 6442 tablet_bootstrap.cc:492] T fd68c5c05f5948b981de64b6785da41a P 145cdcf7953e4c0d9686b1c6a86f9482: No bootstrap required, opened a new log
I20260925 20:00:51.926229 6442 ts_tablet_manager.cc:1403] T fd68c5c05f5948b981de64b6785da41a P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.001s
I20260925 20:00:51.927122 6442 raft_consensus.cc:359] T fd68c5c05f5948b981de64b6785da41a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:51.927439 6442 raft_consensus.cc:385] T fd68c5c05f5948b981de64b6785da41a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 20:00:51.927537 6442 raft_consensus.cc:740] T fd68c5c05f5948b981de64b6785da41a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145cdcf7953e4c0d9686b1c6a86f9482, State: Initialized, Role: FOLLOWER
I20260925 20:00:51.927806 6442 consensus_queue.cc:260] T fd68c5c05f5948b981de64b6785da41a P 145cdcf7953e4c0d9686b1c6a86f9482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:51.928427 4573 heartbeater.cc:499] Master 127.3.199.253:41897 was elected leader, sending a full tablet report...
I20260925 20:00:51.930043 6442 ts_tablet_manager.cc:1434] T fd68c5c05f5948b981de64b6785da41a P 145cdcf7953e4c0d9686b1c6a86f9482: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260925 20:00:51.932096 6439 ts_tablet_manager.cc:1929] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:51.932248 6439 log.cc:1199] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/b59f74f459c542a88e8bef5f1c51357d
I20260925 20:00:51.934028 6439 ts_tablet_manager.cc:1950] T b59f74f459c542a88e8bef5f1c51357d P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:51.935024 6437 tablet_replica.cc:333] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:51.935123 6439 tablet_replica.cc:333] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482: stopping tablet replica
I20260925 20:00:51.935228 6437 raft_consensus.cc:2243] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:51.935339 6439 raft_consensus.cc:2243] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:51.935467 6437 raft_consensus.cc:2272] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:51.935566 6439 raft_consensus.cc:2272] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:51.936144 6437 ts_tablet_manager.cc:1916] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:51.936532 6439 ts_tablet_manager.cc:1916] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:51.939815 6440 ts_tablet_manager.cc:1929] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:51.939991 6440 log.cc:1199] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/b321f36472cc42babf1ec3f64ab990ed
I20260925 20:00:51.940487 6440 ts_tablet_manager.cc:1950] T b321f36472cc42babf1ec3f64ab990ed P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:51.943513 6439 ts_tablet_manager.cc:1929] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:51.943656 6439 log.cc:1199] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-0/wal/wals/83c0e7bd6ef040b9af169d9677137836
I20260925 20:00:51.944129 6439 ts_tablet_manager.cc:1950] T 83c0e7bd6ef040b9af169d9677137836 P 145cdcf7953e4c0d9686b1c6a86f9482: Deleting consensus metadata
I20260925 20:00:51.945359 6440 tablet_replica.cc:333] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:51.945573 6440 raft_consensus.cc:2243] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:51.945788 6440 raft_consensus.cc:2272] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:51.946396 6440 ts_tablet_manager.cc:1916] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:51.948330 6437 ts_tablet_manager.cc:1929] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 20:00:51.948478 6437 log.cc:1199] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/b321f36472cc42babf1ec3f64ab990ed
I20260925 20:00:51.948916 6437 ts_tablet_manager.cc:1950] T b321f36472cc42babf1ec3f64ab990ed P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:51.950119 6437 tablet_replica.cc:333] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:51.950341 6437 raft_consensus.cc:2243] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:51.950699 6437 raft_consensus.cc:2272] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:51.951401 6437 ts_tablet_manager.cc:1916] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:51.953300 6440 ts_tablet_manager.cc:1929] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:51.953447 6440 log.cc:1199] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/81d09be24f0a4687a1b17ddd02fd4e47
I20260925 20:00:51.953881 6440 ts_tablet_manager.cc:1950] T 81d09be24f0a4687a1b17ddd02fd4e47 P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:51.954939 6440 tablet_replica.cc:333] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:51.955185 6440 raft_consensus.cc:2243] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:51.955366 6440 raft_consensus.cc:2272] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:51.956090 6440 ts_tablet_manager.cc:1916] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:51.962868 6440 ts_tablet_manager.cc:1929] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:51.963054 6440 log.cc:1199] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/83c0e7bd6ef040b9af169d9677137836
I20260925 20:00:51.963127 6437 ts_tablet_manager.cc:1929] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:51.963294 6437 log.cc:1199] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/83c0e7bd6ef040b9af169d9677137836
I20260925 20:00:51.963455 6440 ts_tablet_manager.cc:1950] T 83c0e7bd6ef040b9af169d9677137836 P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
I20260925 20:00:51.963680 6437 ts_tablet_manager.cc:1950] T 83c0e7bd6ef040b9af169d9677137836 P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:51.964658 6440 tablet_replica.cc:333] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0: stopping tablet replica
I20260925 20:00:51.964869 6437 tablet_replica.cc:333] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e: stopping tablet replica
I20260925 20:00:51.964861 6440 raft_consensus.cc:2243] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 20:00:51.965070 6440 raft_consensus.cc:2272] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:51.965103 6437 raft_consensus.cc:2243] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [term 1 LEADER]: Raft consensus shutting down.
I20260925 20:00:51.965474 6437 raft_consensus.cc:2272] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 20:00:51.965711 6440 ts_tablet_manager.cc:1916] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:51.966030 6437 ts_tablet_manager.cc:1916] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 20:00:51.970382 6437 ts_tablet_manager.cc:1929] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:51.970541 6437 log.cc:1199] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-2/wal/wals/b59f74f459c542a88e8bef5f1c51357d
I20260925 20:00:51.970912 6437 ts_tablet_manager.cc:1950] T b59f74f459c542a88e8bef5f1c51357d P dd2623c3b2c3497ca1df4a676fa6807e: Deleting consensus metadata
I20260925 20:00:51.972630 6440 ts_tablet_manager.cc:1929] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 20:00:51.972782 6440 log.cc:1199] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0: Deleting WAL directory at /tmp/dist-test-taskg5SBXY/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790366397272667-3871-0/minicluster-data/ts-1/wal/wals/b59f74f459c542a88e8bef5f1c51357d
I20260925 20:00:51.973256 6440 ts_tablet_manager.cc:1950] T b59f74f459c542a88e8bef5f1c51357d P 46a6f571dfa749b7905d8458d4c929b0: Deleting consensus metadata
W20260925 20:00:52.008607 4574 tablet.cc:2433] T fd68c5c05f5948b981de64b6785da41a P 145cdcf7953e4c0d9686b1c6a86f9482: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 20:00:52.140663 6413 raft_consensus.cc:493] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 20:00:52.140967 6413 raft_consensus.cc:515] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
W20260925 20:00:52.143270 4456 connection.cc:573] server connection from 127.0.0.1:58370 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:52.143714 4741 connection.cc:573] server connection from 127.0.0.1:39536 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 20:00:52.143770 6413 leader_election.cc:290] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
W20260925 20:00:52.144691 4602 connection.cc:573] client connection to 127.3.199.253:41897 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:52.144654 4456 connection.cc:573] client connection to 127.3.199.253:41897 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 20:00:52.144797 4603 connection.cc:573] server connection from 127.0.0.1:36778 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 20:00:52.145594 4524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd68c5c05f5948b981de64b6785da41a" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" is_pre_election: true
I20260925 20:00:52.146004 4524 raft_consensus.cc:2468] T fd68c5c05f5948b981de64b6785da41a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 0.
W20260925 20:00:52.146687 4862 connection.cc:573] client connection to 127.3.199.253:41897 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 20:00:52.147327 4603 leader_election.cc:304] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters:
W20260925 20:00:52.147401 4862 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.3.199.253:41897)
I20260925 20:00:52.147686 6413 raft_consensus.cc:2804] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 20:00:52.147802 6413 raft_consensus.cc:493] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 20:00:52.147915 6413 raft_consensus.cc:3060] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [term 0 FOLLOWER]: Advancing to term 1
W20260925 20:00:52.148097 4862 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.3.199.253:41897)
W20260925 20:00:52.148365 4862 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.199.253:41897)
W20260925 20:00:52.148343 4603 leader_election.cc:343] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395): Not found: Tablet not found: fd68c5c05f5948b981de64b6785da41a
W20260925 20:00:52.148883 4862 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.3.199.253:41897)
I20260925 20:00:52.151315 6413 raft_consensus.cc:515] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:52.152597 6413 leader_election.cc:290] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 election: Requested vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395)
I20260925 20:00:52.153162 4524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd68c5c05f5948b981de64b6785da41a" candidate_uuid: "46a6f571dfa749b7905d8458d4c929b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145cdcf7953e4c0d9686b1c6a86f9482"
I20260925 20:00:52.153430 4524 raft_consensus.cc:3060] T fd68c5c05f5948b981de64b6785da41a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 0 FOLLOWER]: Advancing to term 1
W20260925 20:00:52.153683 4603 leader_election.cc:343] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395): Not found: Tablet not found: fd68c5c05f5948b981de64b6785da41a
I20260925 20:00:52.155664 4524 raft_consensus.cc:2468] T fd68c5c05f5948b981de64b6785da41a P 145cdcf7953e4c0d9686b1c6a86f9482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a6f571dfa749b7905d8458d4c929b0 in term 1.
I20260925 20:00:52.156185 4603 leader_election.cc:304] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0; no voters: dd2623c3b2c3497ca1df4a676fa6807e
I20260925 20:00:52.156678 6413 raft_consensus.cc:2804] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 20:00:52.157281 6413 raft_consensus.cc:697] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [term 1 LEADER]: Becoming Leader. State: Replica: 46a6f571dfa749b7905d8458d4c929b0, State: Running, Role: LEADER
W20260925 20:00:52.158208 4715 tablet.cc:2433] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 20:00:52.158104 6413 consensus_queue.cc:237] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:52.354909 6456 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:52.355165 6456 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:52.356225 6456 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:52.359200 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:52.359416 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:52.359557 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:52.359879 6456 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:00:52.856215 3871 master-stress-test.cc:579] Killing test thread 1/25
W20260925 20:00:52.939630 4603 consensus_peers.cc:597] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 -> Peer dd2623c3b2c3497ca1df4a676fa6807e (127.3.199.195:34395): Couldn't send request to peer dd2623c3b2c3497ca1df4a676fa6807e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fd68c5c05f5948b981de64b6785da41a. This is attempt 1: this message will repeat every 5th retry.
I20260925 20:00:53.036595 6465 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:53.036814 6465 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:53.037606 6465 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:53.039896 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:53.040113 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:53.040261 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:53.040742 6465 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:00:53.068233 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:00:53.068516 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:00:53.166420 6413 consensus_queue.cc:1048] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 20:00:53.726502 6481 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:53.726747 6481 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:53.727468 6481 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:53.728639 6032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260925 20:00:53.730005 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:53.730332 6485 ts_tablet_manager.cc:933] T fd68c5c05f5948b981de64b6785da41a P dd2623c3b2c3497ca1df4a676fa6807e: Initiating tablet copy from peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:00:53.730464 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:53.730614 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:53.730888 6481 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:00:53.731360 6485 tablet_copy_client.cc:323] T fd68c5c05f5948b981de64b6785da41a P dd2623c3b2c3497ca1df4a676fa6807e: tablet copy: Beginning tablet copy session from remote peer at address 127.3.199.194:36647
I20260925 20:00:53.732563 4687 tablet_copy_service.cc:140] P 46a6f571dfa749b7905d8458d4c929b0: Received BeginTabletCopySession request for tablet fd68c5c05f5948b981de64b6785da41a from peer dd2623c3b2c3497ca1df4a676fa6807e ({username='slave'} at 127.3.199.195:54497)
I20260925 20:00:53.732796 4687 tablet_copy_service.cc:161] P 46a6f571dfa749b7905d8458d4c929b0: Beginning new tablet copy session on tablet fd68c5c05f5948b981de64b6785da41a from peer dd2623c3b2c3497ca1df4a676fa6807e at {username='slave'} at 127.3.199.195:54497: session id = dd2623c3b2c3497ca1df4a676fa6807e-fd68c5c05f5948b981de64b6785da41a
I20260925 20:00:53.735743 4687 tablet_copy_source_session.cc:215] T fd68c5c05f5948b981de64b6785da41a P 46a6f571dfa749b7905d8458d4c929b0: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 20:00:53.738091 6485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd68c5c05f5948b981de64b6785da41a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 20:00:53.742998 6485 tablet_copy_client.cc:806] T fd68c5c05f5948b981de64b6785da41a P dd2623c3b2c3497ca1df4a676fa6807e: tablet copy: Starting download of 0 data blocks...
I20260925 20:00:53.743361 6485 tablet_copy_client.cc:670] T fd68c5c05f5948b981de64b6785da41a P dd2623c3b2c3497ca1df4a676fa6807e: tablet copy: Starting download of 1 WAL segments...
I20260925 20:00:53.746171 6485 tablet_copy_client.cc:538] T fd68c5c05f5948b981de64b6785da41a P dd2623c3b2c3497ca1df4a676fa6807e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 20:00:53.749594 6485 tablet_bootstrap.cc:492] T fd68c5c05f5948b981de64b6785da41a P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap starting.
I20260925 20:00:53.755308 6485 tablet_bootstrap.cc:492] T fd68c5c05f5948b981de64b6785da41a P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 20:00:53.755779 6485 tablet_bootstrap.cc:492] T fd68c5c05f5948b981de64b6785da41a P dd2623c3b2c3497ca1df4a676fa6807e: Bootstrap complete.
I20260925 20:00:53.756053 6485 ts_tablet_manager.cc:1403] T fd68c5c05f5948b981de64b6785da41a P dd2623c3b2c3497ca1df4a676fa6807e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260925 20:00:53.756850 6485 raft_consensus.cc:359] T fd68c5c05f5948b981de64b6785da41a P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:53.757014 6485 raft_consensus.cc:740] T fd68c5c05f5948b981de64b6785da41a P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2623c3b2c3497ca1df4a676fa6807e, State: Initialized, Role: FOLLOWER
I20260925 20:00:53.757318 6485 consensus_queue.cc:260] T fd68c5c05f5948b981de64b6785da41a P dd2623c3b2c3497ca1df4a676fa6807e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:53.757900 6485 ts_tablet_manager.cc:1434] T fd68c5c05f5948b981de64b6785da41a P dd2623c3b2c3497ca1df4a676fa6807e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 20:00:53.759341 4687 tablet_copy_service.cc:342] P 46a6f571dfa749b7905d8458d4c929b0: Request end of tablet copy session dd2623c3b2c3497ca1df4a676fa6807e-fd68c5c05f5948b981de64b6785da41a received from {username='slave'} at 127.3.199.195:54497
I20260925 20:00:53.759619 4687 tablet_copy_service.cc:434] P 46a6f571dfa749b7905d8458d4c929b0: ending tablet copy session dd2623c3b2c3497ca1df4a676fa6807e-fd68c5c05f5948b981de64b6785da41a on tablet fd68c5c05f5948b981de64b6785da41a with peer dd2623c3b2c3497ca1df4a676fa6807e
I20260925 20:00:54.153095 6488 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:00:54.153327 6488 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:00:54.154141 6488 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:00:54.155437 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:00:54.155896 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:00:54.156030 4739 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:00:54.156452 6488 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:00:54.566417 6488 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:00:54.566920 6488 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:00:54.568109 6488 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:00:54.569326 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:00:54.569900 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:00:54.570128 4739 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:00:54.570472 6488 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:00:54.844787 4803 raft_consensus.cc:1217] T fd68c5c05f5948b981de64b6785da41a P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20260925 20:00:55.025067 6503 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:55.025326 6503 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:55.026100 6503 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:55.028708 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:55.028920 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:55.029052 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:55.029383 6503 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260925 20:00:57.160250 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 6: this message will repeat every 5th retry.
I20260925 20:00:57.330610 6522 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:57.330900 6522 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:57.331677 6522 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:57.334549 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:00:57.334764 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:00:57.334936 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:57.335263 6522 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260925 20:00:57.709635 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 6: this message will repeat every 5th retry.
W20260925 20:00:57.835587 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 6: this message will repeat every 5th retry.
I20260925 20:00:58.072441 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:00:58.072678 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:00:58.082926 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 6: this message will repeat every 5th retry.
I20260925 20:00:59.219937 6546 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:00:59.220207 6546 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:00:59.221055 6546 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:00:59.222143 6031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260925 20:00:59.223727 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:00:59.223948 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:00:59.224052 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:00:59.224272 6546 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:01:00.582732 6540 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:01:00.582969 6540 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:01:00.583704 6540 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:01:00.584797 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:01:00.584957 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:01:00.585136 4739 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:01:00.585431 6540 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:01:00.687637 6540 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:01:00.687873 6540 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:01:00.688962 6540 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:01:00.689524 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:01:00.689894 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:01:00.690006 4739 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:01:00.690280 6540 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:01:02.448395 6584 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:01:02.448634 6584 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:01:02.449373 6584 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:01:02.451889 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:01:02.452090 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:01:02.452224 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:01:02.452638 6584 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:01:03.075822 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:01:03.076071 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:01:05.576999 6558 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:01:05.577255 6558 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:01:05.578013 6558 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:01:05.579058 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:01:05.579263 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:01:05.579442 4739 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:01:05.579727 6558 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:01:07.445813 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 11: this message will repeat every 5th retry.
W20260925 20:01:07.491163 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 11: this message will repeat every 5th retry.
W20260925 20:01:07.544131 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 11: this message will repeat every 5th retry.
I20260925 20:01:08.079700 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:01:08.079913 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:01:08.452224 6641 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:01:08.452512 6641 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:01:08.453299 6641 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:01:08.454471 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:01:08.454828 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:01:08.454988 4741 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:01:08.455308 6641 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:01:08.615803 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 11: this message will repeat every 5th retry.
I20260925 20:01:09.950220 6678 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:01:09.950454 6678 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:01:09.951315 6678 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:01:09.952544 6031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260925 20:01:09.953979 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:01:09.954157 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:01:09.954277 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:01:09.954599 6678 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:01:13.083760 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:01:13.084008 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:01:14.546882 6740 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:01:14.547194 6740 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:01:14.548033 6740 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:01:14.550851 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:01:14.551090 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:01:14.551231 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:01:14.551568 6740 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:01:14.925320 6736 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:01:14.925642 6736 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:01:14.926649 6736 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:01:14.927937 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:01:14.928207 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:01:14.928395 4741 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:01:14.928712 6736 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:01:17.043211 6735 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:01:17.043476 6735 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:01:17.044323 6735 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:01:17.045303 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:01:17.045461 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:01:17.045632 4739 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:01:17.046005 6735 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:01:17.467211 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 16: this message will repeat every 5th retry.
W20260925 20:01:17.546027 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 16: this message will repeat every 5th retry.
W20260925 20:01:18.005959 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 16: this message will repeat every 5th retry.
I20260925 20:01:18.087661 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:01:18.087936 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:01:19.926394 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 16: this message will repeat every 5th retry.
W20260925 20:01:20.777700 4852 heartbeater.cc:646] Failed to heartbeat to 127.3.199.253:41897 (131 consecutive failures): Network error: Failed to ping master at 127.3.199.253:41897: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:01:20.780905 4571 heartbeater.cc:646] Failed to heartbeat to 127.3.199.252:37957 (144 consecutive failures): Network error: Failed to ping master at 127.3.199.252:37957: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:01:20.952607 4712 heartbeater.cc:646] Failed to heartbeat to 127.3.199.252:37957 (145 consecutive failures): Network error: Failed to ping master at 127.3.199.252:37957: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:01:23.091753 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:01:23.092036 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:01:25.271198 6861 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:01:25.271441 6861 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:01:25.272418 6861 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:01:25.273633 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:01:25.273835 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:01:25.273958 4741 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:01:25.274302 6861 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:01:27.121425 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 21: this message will repeat every 5th retry.
W20260925 20:01:27.344519 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 21: this message will repeat every 5th retry.
W20260925 20:01:27.937508 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 21: this message will repeat every 5th retry.
I20260925 20:01:28.095675 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:01:28.095916 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:01:28.708103 6889 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:01:28.708413 6889 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:01:28.709282 6889 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:01:28.710461 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:01:28.710661 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:01:28.710765 4741 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:01:28.711004 6889 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:01:29.824666 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 21: this message will repeat every 5th retry.
I20260925 20:01:29.913340 6934 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:01:29.913573 6934 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:01:29.914661 6934 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:01:29.915925 6032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260925 20:01:29.917693 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:01:29.917959 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:01:29.918092 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:01:29.918502 6934 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:01:33.099748 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:01:33.100000 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:01:34.469702 6994 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:01:34.469964 6994 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:01:34.470875 6994 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:01:34.474881 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:01:34.475178 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:01:34.475303 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:01:34.475759 6994 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260925 20:01:36.787940 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 26: this message will repeat every 5th retry.
W20260925 20:01:37.415340 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 26: this message will repeat every 5th retry.
I20260925 20:01:37.679808 7036 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:01:37.680006 7036 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:01:37.680996 7036 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:01:37.682296 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:01:37.682472 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:01:37.682643 4739 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:01:37.682962 7036 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:01:38.103571 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:01:38.103830 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:01:38.123704 7037 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:01:38.123997 7037 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:01:38.126518 7037 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:01:38.128369 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:01:38.128623 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:01:38.128793 4739 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:01:38.129060 7037 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:01:38.409065 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 26: this message will repeat every 5th retry.
W20260925 20:01:39.592932 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 26: this message will repeat every 5th retry.
I20260925 20:01:43.107565 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:01:43.107836 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:01:45.917878 7139 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:01:45.918104 7139 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:01:45.919077 7139 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:01:45.920406 6032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260925 20:01:45.921844 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:01:45.922066 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:01:45.922185 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:01:45.922573 7139 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260925 20:01:46.563442 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 31: this message will repeat every 5th retry.
W20260925 20:01:46.843765 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 31: this message will repeat every 5th retry.
W20260925 20:01:48.037160 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 31: this message will repeat every 5th retry.
I20260925 20:01:48.110970 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:01:48.111153 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:01:49.166895 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 31: this message will repeat every 5th retry.
I20260925 20:01:49.640254 7037 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:01:49.640586 7037 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:01:49.641413 7037 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:01:49.642343 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:01:49.642495 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:01:49.642664 4741 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:01:49.643014 7037 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:01:51.119266 7037 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:01:51.119477 7037 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:01:51.120417 7037 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:01:51.121307 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:01:51.121519 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:01:51.121646 4739 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:01:51.121937 7037 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:01:53.114845 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:01:53.115062 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:01:55.888836 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 36: this message will repeat every 5th retry.
I20260925 20:01:56.533367 7237 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:01:56.533577 7237 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:01:56.534538 7237 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:01:56.535563 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:01:56.535696 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:01:56.535904 4739 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:01:56.536185 7237 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:01:57.069640 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 36: this message will repeat every 5th retry.
I20260925 20:01:57.270864 7283 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:01:57.271140 7283 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:01:57.271893 7283 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:01:57.273177 6031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260925 20:01:57.274523 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:01:57.274695 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:01:57.274821 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:01:57.275112 7283 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260925 20:01:57.695050 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 36: this message will repeat every 5th retry.
I20260925 20:01:58.117847 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:01:58.118095 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:01:58.399040 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 36: this message will repeat every 5th retry.
I20260925 20:01:59.044487 7293 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:01:59.044768 7293 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:01:59.045459 7293 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:01:59.046573 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:01:59.046777 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:01:59.046957 4741 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:01:59.047284 7293 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:02:03.121500 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:02:03.121786 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:02:06.185719 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 41: this message will repeat every 5th retry.
W20260925 20:02:07.017501 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 41: this message will repeat every 5th retry.
W20260925 20:02:07.539542 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 41: this message will repeat every 5th retry.
I20260925 20:02:08.125439 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:02:08.125713 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:02:08.517073 7423 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:02:08.517309 7423 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:02:08.518093 7423 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:02:08.519210 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:02:08.519371 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:02:08.519510 4741 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:02:08.519768 7423 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:02:08.536855 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 41: this message will repeat every 5th retry.
I20260925 20:02:10.435454 7470 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:02:10.435707 7470 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:02:10.436726 7470 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:02:10.438102 6031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260925 20:02:10.439631 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:02:10.439790 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:02:10.439966 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:02:10.440310 7470 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:02:13.129115 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:02:13.129323 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:02:15.421015 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 46: this message will repeat every 5th retry.
I20260925 20:02:16.590538 7535 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:02:16.590740 7535 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:02:16.591595 7535 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:02:16.592808 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:02:16.593081 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:02:16.593281 4739 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:02:16.593595 7535 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:02:17.447638 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 46: this message will repeat every 5th retry.
W20260925 20:02:17.537586 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 46: this message will repeat every 5th retry.
I20260925 20:02:17.639336 7550 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:02:17.639571 7550 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:02:17.640378 7550 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:02:17.641522 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:02:17.641726 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:02:17.641885 4741 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:02:17.642207 7550 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:02:18.132879 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:02:18.133191 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:02:18.369879 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 46: this message will repeat every 5th retry.
W20260925 20:02:20.881384 4852 heartbeater.cc:646] Failed to heartbeat to 127.3.199.253:41897 (402 consecutive failures): Network error: Failed to ping master at 127.3.199.253:41897: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:02:21.003695 4573 heartbeater.cc:646] Failed to heartbeat to 127.3.199.253:41897 (180 consecutive failures): Network error: Failed to ping master at 127.3.199.253:41897: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:02:21.175684 4714 heartbeater.cc:646] Failed to heartbeat to 127.3.199.253:41897 (182 consecutive failures): Network error: Failed to ping master at 127.3.199.253:41897: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:02:22.827056 7632 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:02:22.827265 7632 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:02:22.828172 7632 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:02:22.829416 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:02:22.829648 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:02:22.829754 4741 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:02:22.830067 7632 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:02:23.136427 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:02:23.136652 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:02:25.313097 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 51: this message will repeat every 5th retry.
W20260925 20:02:27.422068 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 51: this message will repeat every 5th retry.
W20260925 20:02:28.048418 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 51: this message will repeat every 5th retry.
I20260925 20:02:28.140060 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:02:28.140259 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:02:28.272390 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 51: this message will repeat every 5th retry.
I20260925 20:02:28.677977 7722 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:02:28.678340 7722 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:02:28.679380 7722 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:02:28.680822 6032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260925 20:02:28.682449 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:02:28.682677 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:02:28.682849 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:02:28.683272 7722 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:02:31.723008 7766 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:02:31.723244 7766 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:02:31.724180 7766 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:02:31.725248 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:02:31.725406 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:02:31.725570 4741 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:02:31.725907 7766 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:02:32.874344 7782 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:02:32.874563 7782 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:02:32.875425 7782 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:02:32.876659 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:02:32.876928 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:02:32.877185 4739 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:02:32.877585 7782 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:02:33.144230 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:02:33.144588 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:02:35.298805 7828 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:02:35.299113 7828 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:02:35.300259 7828 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:02:35.301545 6031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260925 20:02:35.303964 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:02:35.304262 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:02:35.304525 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:02:35.304857 7828 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260925 20:02:35.863879 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 56: this message will repeat every 5th retry.
W20260925 20:02:37.853012 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 56: this message will repeat every 5th retry.
I20260925 20:02:38.148504 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:02:38.148820 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:02:38.606943 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 56: this message will repeat every 5th retry.
W20260925 20:02:39.068236 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 56: this message will repeat every 5th retry.
I20260925 20:02:43.067256 6034 maintenance_manager.cc:419] P 169b6de20560440b99cc08af549ce477: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033346
I20260925 20:02:43.140650 6033 maintenance_manager.cc:643] P 169b6de20560440b99cc08af549ce477: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.072s user 0.025s sys 0.011s Metrics: {"bytes_written":23652,"cfile_init":1,"dirs.queue_time_us":517,"dirs.run_cpu_time_us":212,"dirs.run_wall_time_us":1352,"drs_written":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4,"lbm_write_time_us":820,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":59,"thread_start_us":662,"threads_started":2,"wal-append.queue_time_us":444}
I20260925 20:02:43.141952 6034 maintenance_manager.cc:419] P 169b6de20560440b99cc08af549ce477: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4066 bytes on disk
I20260925 20:02:43.142791 6033 maintenance_manager.cc:643] P 169b6de20560440b99cc08af549ce477: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":80,"lbm_reads_lt_1ms":4}
I20260925 20:02:43.152666 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:02:43.152848 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:02:45.478577 7973 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:02:45.478826 7973 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:02:45.479619 7973 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:02:45.481041 6031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260925 20:02:45.482312 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:02:45.482595 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:02:45.482762 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:02:45.483042 7973 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260925 20:02:46.171145 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 61: this message will repeat every 5th retry.
I20260925 20:02:47.515810 7993 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:02:47.516047 7993 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:02:47.517020 7993 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:02:47.518345 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:02:47.518599 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:02:47.518694 4739 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:02:47.518965 7993 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:02:47.930008 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 61: this message will repeat every 5th retry.
W20260925 20:02:47.983290 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 61: this message will repeat every 5th retry.
I20260925 20:02:48.155736 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:02:48.155908 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:02:48.924821 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 61: this message will repeat every 5th retry.
I20260925 20:02:50.935135 8008 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:02:50.935314 8008 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:02:50.936161 8008 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:02:50.937436 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:02:50.937633 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:02:50.937762 4739 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:02:50.938035 8008 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:02:52.155200 7993 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:02:52.155417 7993 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:02:52.156203 7993 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:02:52.157310 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:02:52.157455 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:02:52.157603 4741 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:02:52.157927 7993 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:02:53.159809 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:02:53.160035 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:02:56.292994 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 66: this message will repeat every 5th retry.
W20260925 20:02:57.521497 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 66: this message will repeat every 5th retry.
W20260925 20:02:57.737432 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 66: this message will repeat every 5th retry.
I20260925 20:02:58.163264 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:02:58.163542 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:02:58.845959 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 66: this message will repeat every 5th retry.
I20260925 20:02:59.181258 8145 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:02:59.181483 8145 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:02:59.182305 8145 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:02:59.183579 6031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260925 20:02:59.185035 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:02:59.185253 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:02:59.185398 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:02:59.185726 8145 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:02:59.932686 8156 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:02:59.932920 8156 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:02:59.933694 8156 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:02:59.934811 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:02:59.934988 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:02:59.935125 4739 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:02:59.935452 8156 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:03:03.166898 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:03:03.167106 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:03:04.024657 8217 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:03:04.024854 8217 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:03:04.025918 8217 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:03:04.026861 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:03:04.027065 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:03:04.027165 4739 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:03:04.027467 8217 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:03:07.060660 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 71: this message will repeat every 5th retry.
I20260925 20:03:07.286995 8275 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:03:07.287258 8275 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:03:07.288077 8275 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:03:07.289055 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:03:07.289243 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:03:07.289393 4739 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:03:07.289649 8275 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:03:07.696785 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 71: this message will repeat every 5th retry.
W20260925 20:03:07.977166 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 71: this message will repeat every 5th retry.
I20260925 20:03:08.160611 8293 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:03:08.160827 8293 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:03:08.161849 8293 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:03:08.162883 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:03:08.163069 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:03:08.163244 4739 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:03:08.163599 8293 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:03:08.172043 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:03:08.172271 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:03:09.137054 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 71: this message will repeat every 5th retry.
I20260925 20:03:11.641422 8292 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:03:11.641660 8292 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:03:11.642325 8292 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:03:11.643692 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:03:11.643940 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:03:11.644073 4741 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:03:11.644445 8292 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:03:11.985926 8350 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:03:11.986116 8350 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:03:11.986984 8350 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:03:11.987952 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:03:11.988127 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:03:11.988291 4739 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:03:11.988590 8350 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:03:13.175743 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:03:13.175963 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:03:14.987555 8407 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:03:14.987761 8407 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:03:14.988585 8407 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:03:14.989943 6031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260925 20:03:14.991333 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:03:14.991531 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:03:14.991673 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:03:14.991971 8407 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260925 20:03:16.641259 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 76: this message will repeat every 5th retry.
W20260925 20:03:17.591199 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 76: this message will repeat every 5th retry.
W20260925 20:03:17.833997 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 76: this message will repeat every 5th retry.
I20260925 20:03:18.179371 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:03:18.179582 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:03:19.418998 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 76: this message will repeat every 5th retry.
W20260925 20:03:20.895947 4850 heartbeater.cc:646] Failed to heartbeat to 127.3.199.252:37957 (774 consecutive failures): Network error: Failed to ping master at 127.3.199.252:37957: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:03:20.895967 4852 heartbeater.cc:646] Failed to heartbeat to 127.3.199.253:41897 (676 consecutive failures): Network error: Failed to ping master at 127.3.199.253:41897: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:03:20.990866 8500 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:03:20.991127 8500 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:03:20.991979 8500 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:03:20.993198 6032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260925 20:03:20.994959 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:03:20.995175 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:03:20.995390 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:03:20.995626 8500 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260925 20:03:21.224577 4571 heartbeater.cc:646] Failed to heartbeat to 127.3.199.252:37957 (384 consecutive failures): Network error: Failed to ping master at 127.3.199.252:37957: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:03:21.400806 4712 heartbeater.cc:646] Failed to heartbeat to 127.3.199.252:37957 (385 consecutive failures): Network error: Failed to ping master at 127.3.199.252:37957: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:03:21.629848 8477 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:03:21.630067 8477 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:03:21.631518 8477 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:03:21.632697 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:03:21.632876 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:03:21.632982 4741 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:03:21.633255 8477 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:03:23.182874 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:03:23.183072 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:03:24.658489 8417 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:03:24.658756 8417 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:03:24.659459 8417 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:03:24.660588 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:03:24.660780 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:03:24.660952 4741 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:03:24.661309 8417 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:03:27.155061 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 81: this message will repeat every 5th retry.
W20260925 20:03:27.317387 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 81: this message will repeat every 5th retry.
W20260925 20:03:27.561321 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 81: this message will repeat every 5th retry.
I20260925 20:03:28.186645 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:03:28.186887 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:03:29.463490 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 81: this message will repeat every 5th retry.
I20260925 20:03:33.189757 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:03:33.189929 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:03:36.958326 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 86: this message will repeat every 5th retry.
I20260925 20:03:37.470353 8779 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:03:37.470624 8779 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:03:37.471978 8779 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:03:37.473184 6031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260925 20:03:37.475033 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:03:37.475238 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:03:37.475409 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:03:37.475713 8779 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260925 20:03:37.709860 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 86: this message will repeat every 5th retry.
I20260925 20:03:37.916129 8788 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:03:37.916385 8788 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:03:37.917657 8788 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:03:37.918694 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:03:37.918897 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:03:37.919032 4739 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:03:37.919313 8788 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:03:37.989032 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 86: this message will repeat every 5th retry.
I20260925 20:03:38.193678 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:03:38.193886 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:03:38.861747 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 86: this message will repeat every 5th retry.
I20260925 20:03:40.861059 8834 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:03:40.861295 8834 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:03:40.862242 8834 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:03:40.865154 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:03:40.865374 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:03:40.865499 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:03:40.865793 8834 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:03:41.913125 8843 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:03:41.913390 8843 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:03:41.914147 8843 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:03:41.915194 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:03:41.915434 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:03:41.915608 4741 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:03:41.915907 8843 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:03:43.197705 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:03:43.197912 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:03:46.003834 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 91: this message will repeat every 5th retry.
W20260925 20:03:47.257963 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 91: this message will repeat every 5th retry.
I20260925 20:03:47.768582 8896 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:03:47.768853 8896 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:03:47.770239 8896 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:03:47.771176 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:03:47.771366 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:03:47.771519 4741 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:03:47.771833 8896 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:03:47.879889 8896 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:03:47.880154 8896 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:03:47.881088 8896 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:03:47.882282 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:03:47.882462 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:03:47.882596 4741 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:03:47.882941 8896 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:03:48.201179 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:03:48.201407 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:03:48.566743 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 91: this message will repeat every 5th retry.
W20260925 20:03:48.720747 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 91: this message will repeat every 5th retry.
I20260925 20:03:53.204747 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:03:53.204941 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:03:55.869216 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 96: this message will repeat every 5th retry.
W20260925 20:03:56.798686 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 96: this message will repeat every 5th retry.
I20260925 20:03:56.950101 9032 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:03:56.950294 9032 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:03:56.951164 9032 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:03:56.952263 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:03:56.952539 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:03:56.952656 4739 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:03:56.952965 9032 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:03:57.300489 9032 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:03:57.300711 9032 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:03:57.301335 9032 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:03:57.302165 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:03:57.302341 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:03:57.302446 4739 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:03:57.302662 9032 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:03:58.208071 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:03:58.208348 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:03:58.310938 9083 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:03:58.311160 9083 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:03:58.312704 9083 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:03:58.313479 6031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260925 20:03:58.315068 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:03:58.315236 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:03:58.315337 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:03:58.315588 9083 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260925 20:03:58.734309 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 96: this message will repeat every 5th retry.
W20260925 20:03:59.440405 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 96: this message will repeat every 5th retry.
I20260925 20:04:03.211262 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:04:03.211511 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:04:06.266592 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 101: this message will repeat every 5th retry.
W20260925 20:04:06.355201 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 101: this message will repeat every 5th retry.
I20260925 20:04:08.214775 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:04:08.215014 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:04:08.234032 9219 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:04:08.234262 9219 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:04:08.235046 9219 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:04:08.236222 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:04:08.236423 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:04:08.236563 4741 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:04:08.236836 9219 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:04:09.415628 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 101: this message will repeat every 5th retry.
W20260925 20:04:09.631121 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 101: this message will repeat every 5th retry.
I20260925 20:04:13.218701 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:04:13.218945 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:04:15.071000 9342 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:04:15.071329 9342 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:04:15.072373 9342 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:04:15.073500 6032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260925 20:04:15.074986 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:04:15.075193 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:04:15.075320 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:04:15.075579 9342 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:04:15.141480 9331 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:04:15.141716 9331 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:04:15.142511 9331 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:04:15.143441 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:04:15.143572 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:04:15.143769 4739 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:04:15.144238 9331 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:04:16.718640 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 106: this message will repeat every 5th retry.
W20260925 20:04:16.819177 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 106: this message will repeat every 5th retry.
I20260925 20:04:18.222390 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:04:18.222609 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:04:19.061218 9350 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:04:19.061406 9350 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:04:19.062075 9350 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:04:19.063200 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:04:19.063373 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:04:19.063468 4741 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:04:19.063731 9350 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:04:19.205346 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 106: this message will repeat every 5th retry.
W20260925 20:04:20.141258 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 106: this message will repeat every 5th retry.
W20260925 20:04:21.026870 4852 heartbeater.cc:646] Failed to heartbeat to 127.3.199.253:41897 (936 consecutive failures): Network error: Failed to ping master at 127.3.199.253:41897: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:04:21.446048 4573 heartbeater.cc:646] Failed to heartbeat to 127.3.199.253:41897 (420 consecutive failures): Network error: Failed to ping master at 127.3.199.253:41897: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:04:21.624370 4712 heartbeater.cc:646] Failed to heartbeat to 127.3.199.252:37957 (505 consecutive failures): Network error: Failed to ping master at 127.3.199.252:37957: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:04:23.226641 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:04:23.226852 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:04:23.551973 9350 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:04:23.552196 9350 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:04:23.552949 9350 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:04:23.554126 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:04:23.554342 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:04:23.554473 4741 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:04:23.554730 9350 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:04:24.741019 9506 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:04:24.741353 9506 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:04:24.742163 9506 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:04:24.743619 6032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260925 20:04:24.745020 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:04:24.745169 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:04:24.745330 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:04:24.745597 9506 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260925 20:04:26.980348 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 111: this message will repeat every 5th retry.
W20260925 20:04:27.108948 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 111: this message will repeat every 5th retry.
I20260925 20:04:28.230666 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:04:28.230906 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:04:29.315798 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 111: this message will repeat every 5th retry.
W20260925 20:04:29.937377 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 111: this message will repeat every 5th retry.
I20260925 20:04:33.234185 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:04:33.234393 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:04:36.758183 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 116: this message will repeat every 5th retry.
I20260925 20:04:36.855152 9709 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:04:36.855427 9709 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:04:36.856402 9709 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:04:36.857662 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:04:36.857801 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:04:36.857946 4741 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:04:36.858232 9709 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:04:37.055142 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 116: this message will repeat every 5th retry.
I20260925 20:04:37.734315 9723 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:04:37.734555 9723 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:04:37.735414 9723 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:04:37.737203 6032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260925 20:04:37.738781 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:04:37.739001 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:04:37.739132 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:04:37.739382 9723 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:04:38.237308 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:04:38.237538 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:04:38.402292 9728 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:04:38.402520 9728 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:04:38.403218 9728 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:04:38.404315 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:04:38.404461 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:04:38.404599 4741 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:04:38.404873 9728 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:04:39.399860 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 116: this message will repeat every 5th retry.
W20260925 20:04:39.646278 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 116: this message will repeat every 5th retry.
I20260925 20:04:43.241021 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:04:43.241212 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:04:46.022004 9855 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:04:46.022195 9855 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:04:46.023250 9855 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:04:46.024456 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:04:46.024647 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:04:46.024807 4741 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:04:46.025094 9855 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:04:47.119748 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 121: this message will repeat every 5th retry.
W20260925 20:04:47.192942 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 121: this message will repeat every 5th retry.
I20260925 20:04:48.245292 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:04:48.245520 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:04:48.264132 9855 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:04:48.264374 9855 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:04:48.265049 9855 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:04:48.266218 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:04:48.266430 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:04:48.266547 4739 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:04:48.266876 9855 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:04:48.793208 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 121: this message will repeat every 5th retry.
W20260925 20:04:49.726917 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 121: this message will repeat every 5th retry.
I20260925 20:04:52.256907 9957 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:04:52.257180 9957 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:04:52.258148 9957 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:04:52.259367 6031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260925 20:04:52.260895 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:04:52.261053 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:04:52.261206 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:04:52.261447 9957 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:04:53.248651 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:04:53.248901 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:04:55.668920 9999 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:04:55.669256 9999 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:04:55.670073 9999 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:04:55.671090 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:04:55.671283 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:04:55.671398 4739 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:04:55.671643 9999 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:04:57.080112 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 126: this message will repeat every 5th retry.
W20260925 20:04:57.174062 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 126: this message will repeat every 5th retry.
I20260925 20:04:58.252967 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:04:58.253203 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:04:58.705889 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 126: this message will repeat every 5th retry.
W20260925 20:04:59.729897 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 126: this message will repeat every 5th retry.
I20260925 20:05:02.029076 10108 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:05:02.029302 10108 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:05:02.030169 10108 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:05:02.031294 6031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260925 20:05:02.032672 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:05:02.032871 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:05:02.032980 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:05:02.033233 10108 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:05:03.256822 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:05:03.257001 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:05:04.529973 10101 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:05:04.530169 10101 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:05:04.530817 10101 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:05:04.531883 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:05:04.532021 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:05:04.532272 4739 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:05:04.532594 10101 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:05:04.780861 10101 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:05:04.781023 10101 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:05:04.781718 10101 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:05:04.782608 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:05:04.782747 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:05:04.782869 4739 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:05:04.783133 10101 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:05:06.531636 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 131: this message will repeat every 5th retry.
W20260925 20:05:07.696419 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 131: this message will repeat every 5th retry.
I20260925 20:05:08.261776 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:05:08.262208 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:05:09.220461 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 131: this message will repeat every 5th retry.
W20260925 20:05:10.926303 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 131: this message will repeat every 5th retry.
I20260925 20:05:10.981739 10240 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:05:10.981959 10240 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:05:10.982770 10240 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:05:10.983942 6031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260925 20:05:10.985431 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:05:10.985617 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:05:10.985749 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:05:10.986022 10240 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:05:13.265295 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:05:13.265512 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:05:13.925568 10216 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:05:13.925742 10216 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:05:13.926563 10216 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:05:13.927488 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:05:13.927646 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:05:13.927755 4739 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:05:13.928020 10216 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:05:16.227044 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 136: this message will repeat every 5th retry.
W20260925 20:05:17.049117 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 136: this message will repeat every 5th retry.
I20260925 20:05:18.268042 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:05:18.268267 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:05:19.628036 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 136: this message will repeat every 5th retry.
W20260925 20:05:20.997519 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 136: this message will repeat every 5th retry.
W20260925 20:05:21.119791 4850 heartbeater.cc:646] Failed to heartbeat to 127.3.199.252:37957 (1305 consecutive failures): Network error: Failed to ping master at 127.3.199.252:37957: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:05:21.671201 4571 heartbeater.cc:646] Failed to heartbeat to 127.3.199.252:37957 (624 consecutive failures): Network error: Failed to ping master at 127.3.199.252:37957: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:05:21.838465 4714 heartbeater.cc:646] Failed to heartbeat to 127.3.199.253:41897 (542 consecutive failures): Network error: Failed to ping master at 127.3.199.253:41897: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:05:23.271378 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:05:23.271545 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:05:23.553906 10438 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:05:23.554106 10438 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:05:23.554828 10438 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:05:23.556073 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:05:23.556244 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:05:23.556360 4739 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:05:23.556667 10438 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:05:25.428404 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 141: this message will repeat every 5th retry.
W20260925 20:05:27.515489 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 141: this message will repeat every 5th retry.
I20260925 20:05:28.041296 10510 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:05:28.041495 10510 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:05:28.042218 10510 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:05:28.043497 6031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260925 20:05:28.044947 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:05:28.045136 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:05:28.045243 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:05:28.045490 10510 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:05:28.274380 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:05:28.274538 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:05:29.517307 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 141: this message will repeat every 5th retry.
I20260925 20:05:30.052444 10540 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:05:30.052644 10540 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:05:30.053313 10540 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:05:30.057962 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:05:30.058228 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:05:30.058341 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:05:30.058584 10540 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260925 20:05:30.903586 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 141: this message will repeat every 5th retry.
I20260925 20:05:31.280587 10487 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:05:31.280807 10487 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:05:31.281744 10487 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:05:31.282780 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:05:31.282959 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:05:31.283047 4741 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:05:31.283250 10487 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:05:31.580937 10487 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:05:31.581188 10487 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:05:31.582000 10487 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:05:31.582908 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:05:31.583047 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:05:31.583212 4739 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:05:31.583475 10487 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:05:33.278110 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:05:33.278307 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:05:35.625042 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 146: this message will repeat every 5th retry.
W20260925 20:05:36.556602 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 146: this message will repeat every 5th retry.
I20260925 20:05:37.716854 10666 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:05:37.717108 10666 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:05:37.718024 10666 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:05:37.719277 6032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260925 20:05:37.720983 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:05:37.721201 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:05:37.721311 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:05:37.721582 10666 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:05:38.281023 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:05:38.281205 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:05:40.092664 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 146: this message will repeat every 5th retry.
I20260925 20:05:40.470412 10704 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:05:40.470574 10704 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:05:40.471360 10704 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:05:40.472193 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:05:40.472385 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:05:40.472498 4739 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:05:40.472754 10704 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:05:40.517886 10718 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:05:40.518095 10718 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:05:40.518884 10718 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:05:40.521014 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
W20260925 20:05:40.521292 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
I20260925 20:05:40.521433 6031 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:05:40.521723 10718 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260925 20:05:41.003206 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 146: this message will repeat every 5th retry.
I20260925 20:05:43.284415 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:05:43.284602 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 20:05:43.515373 10704 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:05:43.515571 10704 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:05:43.516340 10704 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:05:43.517392 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:05:43.517589 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:05:43.517714 4739 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:05:43.517970 10704 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 20:05:44.564112 10704 raft_consensus.cc:493] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:05:44.564414 10704 raft_consensus.cc:515] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } }
I20260925 20:05:44.565291 10704 leader_election.cc:290] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:05:44.566757 4741 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
W20260925 20:05:44.567030 4739 leader_election.cc:343] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: 72c3c64108f348a5ab893e6954d72d77
I20260925 20:05:44.567204 4739 leader_election.cc:304] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:05:44.567629 10704 raft_consensus.cc:2749] T 72c3c64108f348a5ab893e6954d72d77 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:05:45.103046 4741 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 151: this message will repeat every 5th retry.
W20260925 20:05:46.888105 4739 consensus_peers.cc:597] T 465db2a7c1fd48deb8a09d3366cfc04b P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 465db2a7c1fd48deb8a09d3366cfc04b. This is attempt 151: this message will repeat every 5th retry.
I20260925 20:05:48.287886 6011 master_runner.cc:187] Error getting master registration for 127.3.199.253:41897: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111),
I20260925 20:05:48.288166 6011 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 20:05:49.964883 4739 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Couldn't send request to peer 46a6f571dfa749b7905d8458d4c929b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 151: this message will repeat every 5th retry.
I20260925 20:05:50.480898 10864 raft_consensus.cc:493] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145cdcf7953e4c0d9686b1c6a86f9482)
I20260925 20:05:50.481155 10864 raft_consensus.cc:515] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145cdcf7953e4c0d9686b1c6a86f9482" member_type: VOTER last_known_addr { host: "127.3.199.193" port: 34741 } } peers { permanent_uuid: "46a6f571dfa749b7905d8458d4c929b0" member_type: VOTER last_known_addr { host: "127.3.199.194" port: 36647 } } peers { permanent_uuid: "dd2623c3b2c3497ca1df4a676fa6807e" member_type: VOTER last_known_addr { host: "127.3.199.195" port: 34395 } }
I20260925 20:05:50.482107 10864 leader_election.cc:290] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741), 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647)
W20260925 20:05:50.483276 4739 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46a6f571dfa749b7905d8458d4c929b0 (127.3.199.194:36647): Not found: Tablet not found: b65400989b944ca988d6f26838277336
W20260925 20:05:50.483464 4741 leader_election.cc:343] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Not found: Tablet not found: b65400989b944ca988d6f26838277336
I20260925 20:05:50.483579 4741 leader_election.cc:304] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2623c3b2c3497ca1df4a676fa6807e; no voters: 145cdcf7953e4c0d9686b1c6a86f9482, 46a6f571dfa749b7905d8458d4c929b0
I20260925 20:05:50.483911 10864 raft_consensus.cc:2749] T b65400989b944ca988d6f26838277336 P dd2623c3b2c3497ca1df4a676fa6807e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 20:05:51.031628 4741 consensus_peers.cc:597] T e0648fcd521e4db19359c743447246b7 P dd2623c3b2c3497ca1df4a676fa6807e -> Peer 145cdcf7953e4c0d9686b1c6a86f9482 (127.3.199.193:34741): Couldn't send request to peer 145cdcf7953e4c0d9686b1c6a86f9482. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0648fcd521e4db19359c743447246b7. This is attempt 151: this message will repeat every 5th retry.
I20260925 20:05:51.373293 10888 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da29ee550f82456fa7c5ce7ea916909f)
I20260925 20:05:51.373612 10888 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169b6de20560440b99cc08af549ce477" member_type: VOTER last_known_addr { host: "127.3.199.254" port: 41289 } } peers { permanent_uuid: "da29ee550f82456fa7c5ce7ea916909f" member_type: VOTER last_known_addr { host: "127.3.199.253" port: 41897 } } peers { permanent_uuid: "7f43a6a14ef547f2aeb22ce2b334383e" member_type: VOTER last_known_addr { host: "127.3.199.252" port: 37957 } }
I20260925 20:05:51.374353 10888 leader_election.cc:290] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897), 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957)
W20260925 20:05:51.375418 6031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260925 20:05:51.376773 6031 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer da29ee550f82456fa7c5ce7ea916909f (127.3.199.253:41897): Network error: Client connection negotiation failed: client connection to 127.3.199.253:41897: connect: Connection refused (error 111)
W20260925 20:05:51.376973 6032 leader_election.cc:336] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7f43a6a14ef547f2aeb22ce2b334383e (127.3.199.252:37957): Network error: Client connection negotiation failed: client connection to 127.3.199.252:37957: connect: Connection refused (error 111)
I20260925 20:05:51.377090 6032 leader_election.cc:304] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 169b6de20560440b99cc08af549ce477; no voters: 7f43a6a14ef547f2aeb22ce2b334383e, da29ee550f82456fa7c5ce7ea916909f
I20260925 20:05:51.377329 10888 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 169b6de20560440b99cc08af549ce477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260925 20:05:51.959214 6034 maintenance_manager.cc:419] P 169b6de20560440b99cc08af549ce477: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4066 bytes on disk
I20260925 20:05:51.964599 6033 maintenance_manager.cc:643] P 169b6de20560440b99cc08af549ce477: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.005s sys 0.000s Metrics: {"dirs.queue_time_us":419,"dirs.run_cpu_time_us":121,"dirs.run_wall_time_us":122,"thread_start_us":558,"threads_started":2,"wal-append.queue_time_us":357}
W20260925 20:05:52.613495 4862 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
W20260925 20:05:52.613852 4862 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
W20260925 20:05:52.613931 4862 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
W20260925 20:05:52.614035 4862 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
F20260925 20:05:52.624811 4885 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
*** Aborted at 1790366752 (unix time) try "date -d @1790366752" if you are using GNU date ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629271 4869 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aae983110c484d68a3a085faa1688c5e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629271 4869 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aae983110c484d68a3a085faa1688c5e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630591 4870 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_799678007be3498da115944e990a1785 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629271 4869 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aae983110c484d68a3a085faa1688c5e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630591 4870 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_799678007be3498da115944e990a1785 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
20260925 20:05:52.630647 4880 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629271 4869 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aae983110c484d68a3a085faa1688c5e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630591 4870 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_799678007be3498da115944e990a1785 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
20260925 20:05:52.630647 4880 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630767 4874 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f9d13749640c4f11913f7449b48cf6c8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629271 4869 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aae983110c484d68a3a085faa1688c5e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630591 4870 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_799678007be3498da115944e990a1785 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
20260925 20:05:52.630647 4880 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630767 4874 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f9d13749640c4f11913f7449b48cf6c8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630592 4876 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f051ca2de9a64755b465c9b043ce4392 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629271 4869 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aae983110c484d68a3a085faa1688c5e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630591 4870 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_799678007be3498da115944e990a1785 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
20260925 20:05:52.630647 4880 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630767 4874 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f9d13749640c4f11913f7449b48cf6c8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630592 4876 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f051ca2de9a64755b465c9b043ce4392 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.631004 4890 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800000f1f) received by PID 3871 (TID 0x7fabaf61d700) from PID 3871; stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629271 4869 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aae983110c484d68a3a085faa1688c5e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630591 4870 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_799678007be3498da115944e990a1785 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
20260925 20:05:52.630647 4880 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630767 4874 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f9d13749640c4f11913f7449b48cf6c8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630592 4876 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f051ca2de9a64755b465c9b043ce4392 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.631004 4890 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632419 4881 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629271 4869 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aae983110c484d68a3a085faa1688c5e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630591 4870 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_799678007be3498da115944e990a1785 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
20260925 20:05:52.630647 4880 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630767 4874 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f9d13749640c4f11913f7449b48cf6c8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630592 4876 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f051ca2de9a64755b465c9b043ce4392 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.631004 4890 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632419 4881 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632450 4872 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4627e927cd124d049b0fdc22eb15c554 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629271 4869 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aae983110c484d68a3a085faa1688c5e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630591 4870 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_799678007be3498da115944e990a1785 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
20260925 20:05:52.630647 4880 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630767 4874 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f9d13749640c4f11913f7449b48cf6c8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630592 4876 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f051ca2de9a64755b465c9b043ce4392 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.631004 4890 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632419 4881 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632450 4872 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4627e927cd124d049b0fdc22eb15c554 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632586 4886 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632622 *** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629271 4869 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aae983110c484d68a3a085faa1688c5e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630591 4870 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_799678007be3498da115944e990a1785 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
20260925 20:05:52.630647 4880 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630767 4874 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f9d13749640c4f11913f7449b48cf6c8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630592 4876 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f051ca2de9a64755b465c9b043ce4392 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.631004 4890 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632419 4881 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632450 4872 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4627e927cd124d049b0fdc22eb15c554 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632586 4886 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632622 4889 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629271 4869 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aae983110c484d68a3a085faa1688c5e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630591 4870 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_799678007be3498da115944e990a1785 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
20260925 20:05:52.630647 4880 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630767 4874 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f9d13749640c4f11913f7449b48cf6c8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630592 4876 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f051ca2de9a64755b465c9b043ce4392 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.631004 4890 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632419 4881 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632450 4872 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4627e927cd124d049b0fdc22eb15c554 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632586 4886 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632622 4889 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633090 4873 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f19364a66794406806691ab3a701b72 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629271 4869 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aae983110c484d68a3a085faa1688c5e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630591 4870 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_799678007be3498da115944e990a1785 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
20260925 20:05:52.630647 4880 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630767 4874 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f9d13749640c4f11913f7449b48cf6c8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630592 4876 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f051ca2de9a64755b465c9b043ce4392 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.631004 4890 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632419 4881 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632450 4872 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4627e927cd124d049b0fdc22eb15c554 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632586 4886 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632622 4889 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633090 4873 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f19364a66794406806691ab3a701b72 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633157 4868 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6796b08b82a249d8b499df979ebeb52a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629271 4869 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aae983110c484d68a3a085faa1688c5e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630591 4870 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_799678007be3498da115944e990a1785 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
20260925 20:05:52.630647 4880 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630767 4874 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f9d13749640c4f11913f7449b48cf6c8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630592 4876 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f051ca2de9a64755b465c9b043ce4392 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.631004 4890 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632419 4881 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632450 4872 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4627e927cd124d049b0fdc22eb15c554 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632586 4886 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632622 4889 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633090 4873 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f19364a66794406806691ab3a701b72 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633157 4868 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6796b08b82a249d8b499df979ebeb52a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633337 4894 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629271 4869 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aae983110c484d68a3a085faa1688c5e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630591 4870 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_799678007be3498da115944e990a1785 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
20260925 20:05:52.630647 4880 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630767 4874 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f9d13749640c4f11913f7449b48cf6c8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630592 4876 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f051ca2de9a64755b465c9b043ce4392 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.631004 4890 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632419 4881 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632450 4872 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4627e927cd124d049b0fdc22eb15c554 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632586 4886 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632622 4889 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633090 4873 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f19364a66794406806691ab3a701b72 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633157 4868 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6796b08b82a249d8b499df979ebeb52a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633337 4894 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633402 4871 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_d4b20390d75f45c89df2fe5d8498b337 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629271 4869 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aae983110c484d68a3a085faa1688c5e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630591 4870 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_799678007be3498da115944e990a1785 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
20260925 20:05:52.630647 4880 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630767 4874 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f9d13749640c4f11913f7449b48cf6c8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630592 4876 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f051ca2de9a64755b465c9b043ce4392 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.631004 4890 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632419 4881 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632450 4872 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4627e927cd124d049b0fdc22eb15c554 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632586 4886 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632622 4889 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633090 4873 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f19364a66794406806691ab3a701b72 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633157 4868 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6796b08b82a249d8b499df979ebeb52a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633337 4894 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633402 4871 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_d4b20390d75f45c89df2fe5d8498b337 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633715 4878 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629271 4869 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aae983110c484d68a3a085faa1688c5e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630591 4870 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_799678007be3498da115944e990a1785 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
20260925 20:05:52.630647 4880 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630767 4874 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f9d13749640c4f11913f7449b48cf6c8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630592 4876 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f051ca2de9a64755b465c9b043ce4392 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.631004 4890 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632419 4881 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632450 4872 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4627e927cd124d049b0fdc22eb15c554 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632586 4886 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632622 4889 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633090 4873 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f19364a66794406806691ab3a701b72 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633157 4868 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6796b08b82a249d8b499df979ebeb52a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633337 4894 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633402 4871 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_d4b20390d75f45c89df2fe5d8498b337 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633715 4878 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633818 4875 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ef69b46f2522492baaac95731da2bbcb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
F20260925 20:05:52.627260 4893 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627542 4877 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_30a6c0d501b04bfc882cec2f43c06f83 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627734 4887 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627771 4883 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.627912 4888 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.628093 4884 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629113 4879 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.629271 4869 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_aae983110c484d68a3a085faa1688c5e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630591 4870 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_799678007be3498da115944e990a1785 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
20260925 20:05:52.630647 4880 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630767 4874 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f9d13749640c4f11913f7449b48cf6c8 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.630592 4876 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f051ca2de9a64755b465c9b043ce4392 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.631004 4890 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632419 4881 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632450 4872 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4627e927cd124d049b0fdc22eb15c554 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632586 4886 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.632622 4889 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633090 4873 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9f19364a66794406806691ab3a701b72 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633157 4868 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6796b08b82a249d8b499df979ebeb52a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633337 4894 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633402 4871 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_d4b20390d75f45c89df2fe5d8498b337 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633715 4878 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.633818 4875 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ef69b46f2522492baaac95731da2bbcb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)F20260925 20:05:52.634155 4882 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.3.199.254:41289,127.3.199.253:41897,127.3.199.252:37957, num_attempts: 1)
*** Check failure stack trace: ***
@ 0x7fabcaa7c980 (unknown) at ??:0
@ 0x7fabc3817e87 gsignal at ??:0
@ 0x7fabc38197f1 abort at ??:0
@ 0x7fabc6566d5d google::LogMessage::Fail() at ??:0
@ 0x7fabc656ab23 google::LogMessage::SendToLog() at ??:0
@ 0x7fabc656675c google::LogMessage::Flush() at ??:0
@ 0x7fabc6567ee9 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x40982c kudu::MasterStressTest::DeleteTableThread() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:353
@ 0x4464f4 std::__invoke_impl<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:73
@ 0x4463ad std::__invoke<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:95
@ 0x44636a _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:234
@ 0x44631e std::thread::_Invoker<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:243
@ 0x446059 std::thread::_State_impl<>::_M_run() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:186
@ 0x7fabc3ea16df (unknown) at ??:0
@ 0x7fabcaa716db start_thread at ??:0
@ 0x7fabc38fa61f clone at ??:0