Note: This is test shard 3 of 3.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/2
2026-02-04T08:52:25Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-02-04T08:52:25Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskfmwm4U/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskfmwm4U/thirdparty/src/hadoop-3.4.1/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/security.properties
2026-02-04 08:52:31: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskfmwm4U/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskfmwm4U/thirdparty/src/hadoop-3.4.1/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260204 08:52:40.731491 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
/tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:42471
--webserver_interface=127.0.104.190
--webserver_port=0
--builtin_ntp_servers=127.0.104.148:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260204 08:52:41.077575 661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260204 08:52:41.077903 661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260204 08:52:41.078011 661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260204 08:52:41.086970 661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260204 08:52:41.087075 661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260204 08:52:41.087148 661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260204 08:52:41.087213 661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260204 08:52:41.097383 661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:42471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.104.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0efbebd487e7d0c7f21ab0d5a91e0fa70046af83
build type FASTDEBUG
built by None at 04 Feb 2026 08:43:20 UTC on 24a791456cd2
build id 10340
ASAN enabled
I20260204 08:52:41.099105 661 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260204 08:52:41.101102 661 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260204 08:52:41.111264 667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:41.111459 666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:41.111953 669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260204 08:52:41.113246 661 server_base.cc:1060] running on GCE node
I20260204 08:52:41.114003 661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260204 08:52:41.115520 661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260204 08:52:41.116714 661 hybrid_clock.cc:648] HybridClock initialized: now 1770195161116652 us; error 53 us; skew 500 ppm
I20260204 08:52:41.117071 661 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260204 08:52:41.119308 661 webserver.cc:492] Webserver started at http://127.0.104.190:40787/ using document root <none> and password file <none>
I20260204 08:52:41.120098 661 fs_manager.cc:362] Metadata directory not provided
I20260204 08:52:41.120206 661 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260204 08:52:41.120600 661 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
I20260204 08:52:41.123064 661 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/instance:
uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34"
format_stamp: "Formatted at 2026-02-04 08:52:41 on dist-test-slave-vp4h"
I20260204 08:52:41.123878 661 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal/instance:
uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34"
format_stamp: "Formatted at 2026-02-04 08:52:41 on dist-test-slave-vp4h"
I20260204 08:52:41.129681 661 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260204 08:52:41.133773 675 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260204 08:52:41.135298 661 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260204 08:52:41.135475 661 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data,/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34"
format_stamp: "Formatted at 2026-02-04 08:52:41 on dist-test-slave-vp4h"
I20260204 08:52:41.135699 661 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260204 08:52:41.157183 661 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260204 08:52:41.158135 661 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260204 08:52:41.158408 661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260204 08:52:41.180454 661 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.190:42471
I20260204 08:52:41.180492 726 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.190:42471 every 8 connection(s)
I20260204 08:52:41.181979 661 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
I20260204 08:52:41.186463 727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:41.189371 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu as pid 661
I20260204 08:52:41.189705 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal/instance
I20260204 08:52:41.193176 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
/tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:37005
--webserver_interface=127.0.104.189
--webserver_port=0
--builtin_ntp_servers=127.0.104.148:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260204 08:52:41.195017 727 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } has no permanent_uuid. Determining permanent_uuid...
I20260204 08:52:41.218837 727 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } has no permanent_uuid. Determining permanent_uuid...
W20260204 08:52:41.220095 678 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:52:41.221681 727 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:52:41.271035 727 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } attempt: 1
W20260204 08:52:41.273034 727 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:52:41.341313 727 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } attempt: 2
W20260204 08:52:41.343127 727 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:52:41.434418 727 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } attempt: 3
W20260204 08:52:41.436436 727 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:52:41.556842 728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260204 08:52:41.557178 728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260204 08:52:41.557288 728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260204 08:52:41.566509 728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260204 08:52:41.566620 728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260204 08:52:41.566672 728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260204 08:52:41.566721 728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260204 08:52:41.577437 728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:37005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.104.189
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0efbebd487e7d0c7f21ab0d5a91e0fa70046af83
build type FASTDEBUG
built by None at 04 Feb 2026 08:43:20 UTC on 24a791456cd2
build id 10340
ASAN enabled
I20260204 08:52:41.579074 728 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260204 08:52:41.579728 727 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } attempt: 4
I20260204 08:52:41.581353 728 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260204 08:52:41.581820 727 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:52:41.589955 737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:41.589990 736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:41.590143 739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260204 08:52:41.590843 728 server_base.cc:1060] running on GCE node
I20260204 08:52:41.591645 728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260204 08:52:41.593204 728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260204 08:52:41.594420 728 hybrid_clock.cc:648] HybridClock initialized: now 1770195161594375 us; error 45 us; skew 500 ppm
I20260204 08:52:41.594755 728 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260204 08:52:41.597608 728 webserver.cc:492] Webserver started at http://127.0.104.189:43227/ using document root <none> and password file <none>
I20260204 08:52:41.598399 728 fs_manager.cc:362] Metadata directory not provided
I20260204 08:52:41.598515 728 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260204 08:52:41.598870 728 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
I20260204 08:52:41.601526 728 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data/instance:
uuid: "d083ecc9895345e5b06cd00ef0246c71"
format_stamp: "Formatted at 2026-02-04 08:52:41 on dist-test-slave-vp4h"
I20260204 08:52:41.602371 728 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal/instance:
uuid: "d083ecc9895345e5b06cd00ef0246c71"
format_stamp: "Formatted at 2026-02-04 08:52:41 on dist-test-slave-vp4h"
I20260204 08:52:41.608507 728 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260204 08:52:41.612263 745 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260204 08:52:41.613992 728 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260204 08:52:41.614202 728 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data,/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
uuid: "d083ecc9895345e5b06cd00ef0246c71"
format_stamp: "Formatted at 2026-02-04 08:52:41 on dist-test-slave-vp4h"
I20260204 08:52:41.614434 728 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260204 08:52:41.631551 728 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260204 08:52:41.632551 728 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260204 08:52:41.632853 728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260204 08:52:41.655582 728 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.189:37005
I20260204 08:52:41.655601 796 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.189:37005 every 8 connection(s)
I20260204 08:52:41.657215 728 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data/info.pb
I20260204 08:52:41.657567 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu as pid 728
I20260204 08:52:41.657742 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal/instance
I20260204 08:52:41.659097 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
/tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.188:38981
--webserver_interface=127.0.104.188
--webserver_port=0
--builtin_ntp_servers=127.0.104.148:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260204 08:52:41.663504 797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:41.669775 797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } has no permanent_uuid. Determining permanent_uuid...
I20260204 08:52:41.683851 797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } has no permanent_uuid. Determining permanent_uuid...
I20260204 08:52:41.693516 797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } has no permanent_uuid. Determining permanent_uuid...
W20260204 08:52:41.694733 746 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:52:41.696398 797 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:38981: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:52:41.745728 797 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } attempt: 1
W20260204 08:52:41.747592 797 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:38981: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:52:41.815892 797 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } attempt: 2
W20260204 08:52:41.817739 797 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:38981: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:52:41.881134 727 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } attempt: 5
I20260204 08:52:41.887689 727 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } has no permanent_uuid. Determining permanent_uuid...
W20260204 08:52:41.889540 727 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:38981: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:52:41.909003 797 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } attempt: 3
W20260204 08:52:41.910914 797 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:38981: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:52:41.940755 727 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } attempt: 1
W20260204 08:52:41.942579 727 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:38981: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:52:42.010824 727 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } attempt: 2
W20260204 08:52:42.012840 727 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:38981: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:52:42.014115 798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260204 08:52:42.014425 798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260204 08:52:42.014529 798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260204 08:52:42.023345 798 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260204 08:52:42.023455 798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260204 08:52:42.023512 798 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260204 08:52:42.023576 798 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260204 08:52:42.034081 798 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.188:38981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.104.188
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0efbebd487e7d0c7f21ab0d5a91e0fa70046af83
build type FASTDEBUG
built by None at 04 Feb 2026 08:43:20 UTC on 24a791456cd2
build id 10340
ASAN enabled
I20260204 08:52:42.035706 798 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260204 08:52:42.037547 798 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260204 08:52:42.045740 806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:42.045720 809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:42.045686 807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260204 08:52:42.046483 798 server_base.cc:1060] running on GCE node
I20260204 08:52:42.047247 798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260204 08:52:42.048812 798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260204 08:52:42.050017 798 hybrid_clock.cc:648] HybridClock initialized: now 1770195162049961 us; error 55 us; skew 500 ppm
I20260204 08:52:42.050364 798 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260204 08:52:42.053038 798 webserver.cc:492] Webserver started at http://127.0.104.188:41735/ using document root <none> and password file <none>
I20260204 08:52:42.053786 798 fs_manager.cc:362] Metadata directory not provided
I20260204 08:52:42.053894 798 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260204 08:52:42.054131 797 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } attempt: 4
I20260204 08:52:42.054244 798 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
W20260204 08:52:42.055850 797 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:38981: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:52:42.057096 798 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-2/data/instance:
uuid: "c1e719e457e94a5da71929720411ae69"
format_stamp: "Formatted at 2026-02-04 08:52:42 on dist-test-slave-vp4h"
I20260204 08:52:42.057955 798 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-2/wal/instance:
uuid: "c1e719e457e94a5da71929720411ae69"
format_stamp: "Formatted at 2026-02-04 08:52:42 on dist-test-slave-vp4h"
I20260204 08:52:42.063360 798 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260204 08:52:42.066875 815 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260204 08:52:42.068362 798 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260204 08:52:42.068524 798 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-2/data,/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-2/wal
uuid: "c1e719e457e94a5da71929720411ae69"
format_stamp: "Formatted at 2026-02-04 08:52:42 on dist-test-slave-vp4h"
I20260204 08:52:42.068727 798 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260204 08:52:42.106238 798 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260204 08:52:42.107148 798 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260204 08:52:42.107453 798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260204 08:52:42.119071 727 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } attempt: 3
W20260204 08:52:42.121236 727 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.104.188:38981: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:52:42.128690 798 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.188:38981
I20260204 08:52:42.128707 866 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.188:38981 every 8 connection(s)
I20260204 08:52:42.130409 798 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-2/data/info.pb
I20260204 08:52:42.135176 867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:42.137674 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu as pid 798
I20260204 08:52:42.137854 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-2/wal/instance
I20260204 08:52:42.142930 867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } has no permanent_uuid. Determining permanent_uuid...
I20260204 08:52:42.158989 867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } has no permanent_uuid. Determining permanent_uuid...
I20260204 08:52:42.165457 867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } has no permanent_uuid. Determining permanent_uuid...
I20260204 08:52:42.179512 867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: Bootstrap starting.
I20260204 08:52:42.184072 867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:42.185392 867 log.cc:826] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: Log is configured to *not* fsync() on all Append() calls
I20260204 08:52:42.188035 867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: No bootstrap required, opened a new log
I20260204 08:52:42.195122 867 raft_consensus.cc:359] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:42.195459 867 raft_consensus.cc:385] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:42.195600 867 raft_consensus.cc:740] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e719e457e94a5da71929720411ae69, State: Initialized, Role: FOLLOWER
I20260204 08:52:42.196261 867 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:42.197314 875 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:42.197639 875 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:42.198472 867 sys_catalog.cc:565] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [sys.catalog]: configured and running, proceeding with master startup.
I20260204 08:52:42.208978 798 master_runner.cc:186] Error getting master registration for 127.0.104.190:42471: OK, instance_id { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" instance_seqno: 1770195161174680 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260204 08:52:42.215764 798 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260204 08:52:42.217911 889 catalog_manager.cc:1568] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260204 08:52:42.218127 889 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260204 08:52:42.298556 727 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } attempt: 4
I20260204 08:52:42.310487 727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Bootstrap starting.
I20260204 08:52:42.314134 727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:42.315311 727 log.cc:826] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Log is configured to *not* fsync() on all Append() calls
I20260204 08:52:42.318365 727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: No bootstrap required, opened a new log
I20260204 08:52:42.324362 727 raft_consensus.cc:359] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:42.324714 727 raft_consensus.cc:385] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:42.324981 727 raft_consensus.cc:740] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebeeb5d4823b41b3a8e37546ff2f6e34, State: Initialized, Role: FOLLOWER
I20260204 08:52:42.326215 727 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:42.327925 893 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:42.328243 893 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:42.328925 727 sys_catalog.cc:565] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: configured and running, proceeding with master startup.
I20260204 08:52:42.342319 661 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: OK, instance_id { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" instance_seqno: 1770195161649730 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260204 08:52:42.350446 661 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260204 08:52:42.351140 907 catalog_manager.cc:1568] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260204 08:52:42.351291 907 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260204 08:52:42.355082 797 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } attempt: 5
W20260204 08:52:42.359584 821 tablet.cc:2378] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260204 08:52:42.370271 797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Bootstrap starting.
I20260204 08:52:42.374691 797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:42.375849 797 log.cc:826] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Log is configured to *not* fsync() on all Append() calls
I20260204 08:52:42.378489 797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: No bootstrap required, opened a new log
I20260204 08:52:42.384003 797 raft_consensus.cc:359] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:42.384385 797 raft_consensus.cc:385] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:42.384549 797 raft_consensus.cc:740] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d083ecc9895345e5b06cd00ef0246c71, State: Initialized, Role: FOLLOWER
I20260204 08:52:42.385437 797 consensus_queue.cc:260] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:42.386482 911 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:42.386857 911 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:42.387902 797 sys_catalog.cc:565] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: configured and running, proceeding with master startup.
W20260204 08:52:42.411723 681 tablet.cc:2378] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260204 08:52:42.410586 926 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260204 08:52:42.425751 728 master_runner.cc:263] No leader master found from master d083ecc9895345e5b06cd00ef0246c71
I20260204 08:52:42.425915 728 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:52:42.557552 911 raft_consensus.cc:493] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:42.557819 911 raft_consensus.cc:515] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:42.559291 911 leader_election.cc:290] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
I20260204 08:52:42.559880 702 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d083ecc9895345e5b06cd00ef0246c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" is_pre_election: true
I20260204 08:52:42.560396 702 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d083ecc9895345e5b06cd00ef0246c71 in term 0.
I20260204 08:52:42.561100 842 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d083ecc9895345e5b06cd00ef0246c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e719e457e94a5da71929720411ae69" is_pre_election: true
I20260204 08:52:42.561210 747 leader_election.cc:304] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d083ecc9895345e5b06cd00ef0246c71, ebeeb5d4823b41b3a8e37546ff2f6e34; no voters:
I20260204 08:52:42.561728 911 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:42.561856 911 raft_consensus.cc:493] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:42.561925 911 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:42.562180 842 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d083ecc9895345e5b06cd00ef0246c71 in term 0.
I20260204 08:52:42.564350 911 raft_consensus.cc:515] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:42.565125 911 leader_election.cc:290] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 1 election: Requested vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
I20260204 08:52:42.565676 702 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d083ecc9895345e5b06cd00ef0246c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34"
I20260204 08:52:42.565907 702 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:42.566089 842 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d083ecc9895345e5b06cd00ef0246c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e719e457e94a5da71929720411ae69"
I20260204 08:52:42.566285 842 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:42.569034 702 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d083ecc9895345e5b06cd00ef0246c71 in term 1.
I20260204 08:52:42.569636 747 leader_election.cc:304] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d083ecc9895345e5b06cd00ef0246c71, ebeeb5d4823b41b3a8e37546ff2f6e34; no voters:
I20260204 08:52:42.569669 842 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d083ecc9895345e5b06cd00ef0246c71 in term 1.
I20260204 08:52:42.570093 911 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:42.570688 911 raft_consensus.cc:697] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 1 LEADER]: Becoming Leader. State: Replica: d083ecc9895345e5b06cd00ef0246c71, State: Running, Role: LEADER
I20260204 08:52:42.571215 911 consensus_queue.cc:237] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:42.575536 927 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d083ecc9895345e5b06cd00ef0246c71. Latest consensus state: current_term: 1 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:42.575860 927 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: LEADER
I20260204 08:52:42.586521 929 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260204 08:52:42.591818 929 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260204 08:52:42.614121 842 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 1 FOLLOWER]: Refusing update from remote peer d083ecc9895345e5b06cd00ef0246c71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260204 08:52:42.614214 702 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 1 FOLLOWER]: Refusing update from remote peer d083ecc9895345e5b06cd00ef0246c71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260204 08:52:42.615218 927 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:42.618844 927 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:42.633625 875 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d083ecc9895345e5b06cd00ef0246c71. Latest consensus state: current_term: 1 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:42.633900 875 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:42.638209 927 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:42.638571 927 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: LEADER
I20260204 08:52:42.642567 893 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d083ecc9895345e5b06cd00ef0246c71. Latest consensus state: current_term: 1 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:42.642848 893 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:42.643539 929 catalog_manager.cc:1357] Generated new cluster ID: 41c7773c62174f70835c9cfa744e1148
I20260204 08:52:42.643652 929 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260204 08:52:42.646664 927 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:42.646903 927 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: LEADER
I20260204 08:52:42.653437 893 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:42.653684 893 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:42.654035 932 mvcc.cc:204] Tried to move back new op lower bound from 7250719386055270400 to 7250719385900445696. Current Snapshot: MvccSnapshot[applied={T|T < 7250719386055270400 or (T in {7250719386055270400})}]
I20260204 08:52:42.655678 875 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:42.655874 875 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:42.718200 929 catalog_manager.cc:1380] Generated new certificate authority record
I20260204 08:52:42.720220 929 catalog_manager.cc:1514] Loading token signing keys...
I20260204 08:52:42.739797 929 catalog_manager.cc:6027] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Generated new TSK 0
I20260204 08:52:42.741135 929 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260204 08:52:42.742868 929 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260204 08:52:42.744091 929 catalog_manager.cc:5876] Last processed Hive Metastore notification event ID: -1
I20260204 08:52:42.757695 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
/tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.104.129:0
--local_ip_for_outbound_sockets=127.0.104.129
--webserver_interface=127.0.104.129
--webserver_port=0
--tserver_master_addrs=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--builtin_ntp_servers=127.0.104.148:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260204 08:52:43.220343 889 catalog_manager.cc:1269] Loaded cluster ID: 41c7773c62174f70835c9cfa744e1148
I20260204 08:52:43.220439 889 catalog_manager.cc:1562] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: loading cluster ID for follower catalog manager: success
I20260204 08:52:43.223073 889 catalog_manager.cc:1584] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: acquiring CA information for follower catalog manager: success
I20260204 08:52:43.224926 889 catalog_manager.cc:1612] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260204 08:52:43.238998 943 raft_consensus.cc:493] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:52:43.239252 943 raft_consensus.cc:515] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:43.240793 943 leader_election.cc:290] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005)
I20260204 08:52:43.241407 701 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1e719e457e94a5da71929720411ae69" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" is_pre_election: true
I20260204 08:52:43.241544 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1e719e457e94a5da71929720411ae69" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d083ecc9895345e5b06cd00ef0246c71" is_pre_election: true
I20260204 08:52:43.242437 818 leader_election.cc:304] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1e719e457e94a5da71929720411ae69; no voters: d083ecc9895345e5b06cd00ef0246c71, ebeeb5d4823b41b3a8e37546ff2f6e34
I20260204 08:52:43.242815 943 raft_consensus.cc:2749] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260204 08:52:43.244705 939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260204 08:52:43.244987 939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260204 08:52:43.245095 939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260204 08:52:43.253916 939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260204 08:52:43.254127 939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.104.129
I20260204 08:52:43.263942 939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.104.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.104.129
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.104.129
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 0efbebd487e7d0c7f21ab0d5a91e0fa70046af83
build type FASTDEBUG
built by None at 04 Feb 2026 08:43:20 UTC on 24a791456cd2
build id 10340
ASAN enabled
I20260204 08:52:43.265606 939 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260204 08:52:43.267508 939 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260204 08:52:43.277591 945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:43.278184 946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:43.278436 948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260204 08:52:43.279280 939 server_base.cc:1060] running on GCE node
I20260204 08:52:43.279850 939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260204 08:52:43.280866 939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260204 08:52:43.282042 939 hybrid_clock.cc:648] HybridClock initialized: now 1770195163281996 us; error 50 us; skew 500 ppm
I20260204 08:52:43.282387 939 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260204 08:52:43.284686 939 webserver.cc:492] Webserver started at http://127.0.104.129:41497/ using document root <none> and password file <none>
I20260204 08:52:43.285472 939 fs_manager.cc:362] Metadata directory not provided
I20260204 08:52:43.285583 939 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260204 08:52:43.285952 939 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
I20260204 08:52:43.288479 939 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/data/instance:
uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
format_stamp: "Formatted at 2026-02-04 08:52:43 on dist-test-slave-vp4h"
I20260204 08:52:43.289772 939 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/instance:
uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
format_stamp: "Formatted at 2026-02-04 08:52:43 on dist-test-slave-vp4h"
I20260204 08:52:43.295656 939 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260204 08:52:43.299515 954 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260204 08:52:43.301398 939 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260204 08:52:43.301602 939 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal
uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
format_stamp: "Formatted at 2026-02-04 08:52:43 on dist-test-slave-vp4h"
I20260204 08:52:43.301848 939 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260204 08:52:43.315308 939 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260204 08:52:43.316164 939 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260204 08:52:43.316460 939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260204 08:52:43.317838 939 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260204 08:52:43.319597 939 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260204 08:52:43.319685 939 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260204 08:52:43.319785 939 ts_tablet_manager.cc:616] Registered 0 tablets
I20260204 08:52:43.319852 939 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260204 08:52:43.353945 907 catalog_manager.cc:1269] Loaded cluster ID: 41c7773c62174f70835c9cfa744e1148
I20260204 08:52:43.354064 907 catalog_manager.cc:1562] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: loading cluster ID for follower catalog manager: success
I20260204 08:52:43.356266 939 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.129:36675
I20260204 08:52:43.356329 1066 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.129:36675 every 8 connection(s)
I20260204 08:52:43.357347 907 catalog_manager.cc:1584] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: acquiring CA information for follower catalog manager: success
I20260204 08:52:43.357656 939 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/data/info.pb
I20260204 08:52:43.360052 907 catalog_manager.cc:1612] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260204 08:52:43.364478 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu as pid 939
I20260204 08:52:43.364741 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/instance
I20260204 08:52:43.371465 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
/tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.104.130:0
--local_ip_for_outbound_sockets=127.0.104.130
--webserver_interface=127.0.104.130
--webserver_port=0
--tserver_master_addrs=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--builtin_ntp_servers=127.0.104.148:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260204 08:52:43.395287 1069 heartbeater.cc:344] Connected to a master server at 127.0.104.189:37005
I20260204 08:52:43.395943 1068 heartbeater.cc:344] Connected to a master server at 127.0.104.188:38981
I20260204 08:52:43.396142 1068 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:43.396633 1069 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:43.397445 1068 heartbeater.cc:507] Master 127.0.104.188:38981 requested a full tablet report, sending...
I20260204 08:52:43.397464 1069 heartbeater.cc:507] Master 127.0.104.189:37005 requested a full tablet report, sending...
I20260204 08:52:43.399678 1067 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:52:43.399770 762 ts_manager.cc:194] Registered new tserver with Master: 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:43.399849 1067 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:43.399942 832 ts_manager.cc:194] Registered new tserver with Master: 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:43.400490 1067 heartbeater.cc:507] Master 127.0.104.190:42471 requested a full tablet report, sending...
I20260204 08:52:43.402787 692 ts_manager.cc:194] Registered new tserver with Master: 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:43.401721 762 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.104.129:45029
I20260204 08:52:43.624433 1078 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:52:43.624655 1078 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:43.625963 1078 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
I20260204 08:52:43.626719 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d083ecc9895345e5b06cd00ef0246c71" is_pre_election: true
I20260204 08:52:43.626828 841 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c1e719e457e94a5da71929720411ae69" is_pre_election: true
I20260204 08:52:43.627449 676 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:52:43.627863 1078 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260204 08:52:43.760155 1077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260204 08:52:43.760527 1077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260204 08:52:43.760623 1077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260204 08:52:43.768880 1077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260204 08:52:43.769078 1077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.104.130
I20260204 08:52:43.779891 1077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.104.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.104.130
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.104.130
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 0efbebd487e7d0c7f21ab0d5a91e0fa70046af83
build type FASTDEBUG
built by None at 04 Feb 2026 08:43:20 UTC on 24a791456cd2
build id 10340
ASAN enabled
I20260204 08:52:43.781558 1077 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260204 08:52:43.783318 1077 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260204 08:52:43.793111 1084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:43.793582 1083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:43.793701 1086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260204 08:52:43.794266 1077 server_base.cc:1060] running on GCE node
I20260204 08:52:43.794852 1077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260204 08:52:43.795835 1077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260204 08:52:43.797083 1077 hybrid_clock.cc:648] HybridClock initialized: now 1770195163797019 us; error 49 us; skew 500 ppm
I20260204 08:52:43.797426 1077 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260204 08:52:43.799712 1077 webserver.cc:492] Webserver started at http://127.0.104.130:41259/ using document root <none> and password file <none>
I20260204 08:52:43.800577 1077 fs_manager.cc:362] Metadata directory not provided
I20260204 08:52:43.800693 1077 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260204 08:52:43.801062 1077 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
I20260204 08:52:43.803489 1077 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/data/instance:
uuid: "fdea56c2d1f5499087b939ec9f9dc070"
format_stamp: "Formatted at 2026-02-04 08:52:43 on dist-test-slave-vp4h"
I20260204 08:52:43.804279 1077 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/instance:
uuid: "fdea56c2d1f5499087b939ec9f9dc070"
format_stamp: "Formatted at 2026-02-04 08:52:43 on dist-test-slave-vp4h"
I20260204 08:52:43.809877 1077 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260204 08:52:43.814172 1092 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260204 08:52:43.815683 1077 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260204 08:52:43.815867 1077 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal
uuid: "fdea56c2d1f5499087b939ec9f9dc070"
format_stamp: "Formatted at 2026-02-04 08:52:43 on dist-test-slave-vp4h"
I20260204 08:52:43.816076 1077 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260204 08:52:43.843156 1077 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260204 08:52:43.844055 1077 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260204 08:52:43.844362 1077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260204 08:52:43.845870 1077 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260204 08:52:43.847635 1077 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260204 08:52:43.847738 1077 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260204 08:52:43.847842 1077 ts_tablet_manager.cc:616] Registered 0 tablets
I20260204 08:52:43.847908 1077 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260204 08:52:43.882846 1077 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.130:33737
I20260204 08:52:43.883363 1204 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.130:33737 every 8 connection(s)
I20260204 08:52:43.884574 1077 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/data/info.pb
I20260204 08:52:43.889351 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu as pid 1077
I20260204 08:52:43.889564 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/instance
I20260204 08:52:43.897177 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
/tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.104.131:0
--local_ip_for_outbound_sockets=127.0.104.131
--webserver_interface=127.0.104.131
--webserver_port=0
--tserver_master_addrs=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--builtin_ntp_servers=127.0.104.148:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260204 08:52:43.909494 1069 heartbeater.cc:499] Master 127.0.104.189:37005 was elected leader, sending a full tablet report...
I20260204 08:52:43.910146 1205 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:52:43.910511 1205 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:43.910617 1206 heartbeater.cc:344] Connected to a master server at 127.0.104.188:38981
I20260204 08:52:43.910794 1206 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:43.910177 1207 heartbeater.cc:344] Connected to a master server at 127.0.104.189:37005
I20260204 08:52:43.911258 1207 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:43.911695 1206 heartbeater.cc:507] Master 127.0.104.188:38981 requested a full tablet report, sending...
I20260204 08:52:43.911794 1207 heartbeater.cc:507] Master 127.0.104.189:37005 requested a full tablet report, sending...
I20260204 08:52:43.912663 1205 heartbeater.cc:507] Master 127.0.104.190:42471 requested a full tablet report, sending...
I20260204 08:52:43.913411 832 ts_manager.cc:194] Registered new tserver with Master: fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:43.913908 762 ts_manager.cc:194] Registered new tserver with Master: fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:43.914023 692 ts_manager.cc:194] Registered new tserver with Master: fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:43.914916 762 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.104.130:35287
I20260204 08:52:44.192628 1214 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:52:44.192901 1214 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:44.193698 1214 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
I20260204 08:52:44.194348 841 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c1e719e457e94a5da71929720411ae69" is_pre_election: true
I20260204 08:52:44.195163 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d083ecc9895345e5b06cd00ef0246c71" is_pre_election: true
I20260204 08:52:44.195677 678 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:52:44.195971 1214 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260204 08:52:44.272195 1213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260204 08:52:44.272583 1213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260204 08:52:44.272718 1213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260204 08:52:44.281802 1213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260204 08:52:44.282040 1213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.104.131
I20260204 08:52:44.292660 1213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.104.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.104.131
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.104.131
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 0efbebd487e7d0c7f21ab0d5a91e0fa70046af83
build type FASTDEBUG
built by None at 04 Feb 2026 08:43:20 UTC on 24a791456cd2
build id 10340
ASAN enabled
I20260204 08:52:44.294342 1213 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260204 08:52:44.296236 1213 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260204 08:52:44.306797 1219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:44.306870 1222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:44.306774 1220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260204 08:52:44.313541 1213 server_base.cc:1060] running on GCE node
I20260204 08:52:44.314273 1213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260204 08:52:44.315372 1213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260204 08:52:44.316604 1213 hybrid_clock.cc:648] HybridClock initialized: now 1770195164316557 us; error 46 us; skew 500 ppm
I20260204 08:52:44.316951 1213 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260204 08:52:44.319797 1213 webserver.cc:492] Webserver started at http://127.0.104.131:34775/ using document root <none> and password file <none>
I20260204 08:52:44.320695 1213 fs_manager.cc:362] Metadata directory not provided
I20260204 08:52:44.320825 1213 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260204 08:52:44.321237 1213 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
I20260204 08:52:44.323864 1213 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/data/instance:
uuid: "1234acd7d0db405085a5d95795007134"
format_stamp: "Formatted at 2026-02-04 08:52:44 on dist-test-slave-vp4h"
I20260204 08:52:44.324795 1213 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/instance:
uuid: "1234acd7d0db405085a5d95795007134"
format_stamp: "Formatted at 2026-02-04 08:52:44 on dist-test-slave-vp4h"
I20260204 08:52:44.330751 1213 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260204 08:52:44.334600 1228 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260204 08:52:44.336231 1213 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260204 08:52:44.336462 1213 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal
uuid: "1234acd7d0db405085a5d95795007134"
format_stamp: "Formatted at 2026-02-04 08:52:44 on dist-test-slave-vp4h"
I20260204 08:52:44.336683 1213 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260204 08:52:44.378973 1213 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260204 08:52:44.379932 1213 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260204 08:52:44.380211 1213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260204 08:52:44.381264 1213 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260204 08:52:44.383100 1213 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260204 08:52:44.383206 1213 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260204 08:52:44.383389 1213 ts_tablet_manager.cc:616] Registered 0 tablets
I20260204 08:52:44.383512 1213 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260204 08:52:44.417752 1207 heartbeater.cc:499] Master 127.0.104.189:37005 was elected leader, sending a full tablet report...
I20260204 08:52:44.431761 1213 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.131:46491
I20260204 08:52:44.431815 1340 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.131:46491 every 8 connection(s)
I20260204 08:52:44.433542 1213 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/data/info.pb
I20260204 08:52:44.437561 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu as pid 1213
I20260204 08:52:44.437767 418 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/instance
I20260204 08:52:44.455389 1341 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:52:44.455852 1341 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:44.456854 1342 heartbeater.cc:344] Connected to a master server at 127.0.104.188:38981
I20260204 08:52:44.457011 1342 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:44.457170 1341 heartbeater.cc:507] Master 127.0.104.190:42471 requested a full tablet report, sending...
I20260204 08:52:44.457566 1342 heartbeater.cc:507] Master 127.0.104.188:38981 requested a full tablet report, sending...
I20260204 08:52:44.459052 1343 heartbeater.cc:344] Connected to a master server at 127.0.104.189:37005
I20260204 08:52:44.459252 692 ts_manager.cc:194] Registered new tserver with Master: 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:44.459267 832 ts_manager.cc:194] Registered new tserver with Master: 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:44.459535 1343 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:44.460135 1343 heartbeater.cc:507] Master 127.0.104.189:37005 requested a full tablet report, sending...
I20260204 08:52:44.461102 762 ts_manager.cc:194] Registered new tserver with Master: 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:44.462034 762 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.104.131:34107
I20260204 08:52:44.464551 418 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20260204 08:52:44.496245 761 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41804:
name: "default.Table_dd85a76e9cd94cc0b63f44ac7ed48a3a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:44.496075 762 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41804:
name: "default.Table_ea715cabe3754310a06d2ce6559f0d4b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:44.499508 757 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41804:
name: "default.Table_42998faad7dd446c8a35b76a3a9b5994"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:44.496555 760 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41804:
name: "default.Table_3956d1827061406b8d0a913345864f74"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:44.501145 755 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41804:
name: "default.Table_798df4bc6a0640c4b26f8fe2b3444d6b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:44.497689 759 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41804:
name: "default.Table_43ccd0dac1b24e20880a930959aa280a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:44.498905 758 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41804:
name: "default.Table_279632cf491c4198ad9b91a114c3fc33"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:44.504742 754 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41804:
name: "default.Table_73e876d60c444a5daa316e4bf7158614"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:44.499819 756 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41804:
name: "default.Table_b4244312363c4346b110598eb01428a3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:44.507477 753 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41804:
name: "default.Table_9e950ede78084ef18f4e1051d9a42a90"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260204 08:52:44.510334 762 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_ea715cabe3754310a06d2ce6559f0d4b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260204 08:52:44.515357 761 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260204 08:52:44.515618 755 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_798df4bc6a0640c4b26f8fe2b3444d6b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260204 08:52:44.515357 757 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_42998faad7dd446c8a35b76a3a9b5994 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260204 08:52:44.516526 760 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_3956d1827061406b8d0a913345864f74 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260204 08:52:44.517513 759 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_43ccd0dac1b24e20880a930959aa280a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260204 08:52:44.518465 758 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_279632cf491c4198ad9b91a114c3fc33 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260204 08:52:44.519400 754 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_73e876d60c444a5daa316e4bf7158614 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260204 08:52:44.520803 753 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_9e950ede78084ef18f4e1051d9a42a90 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260204 08:52:44.521625 756 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_b4244312363c4346b110598eb01428a3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260204 08:52:44.547983 418 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu pid 661
I20260204 08:52:44.549456 722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260204 08:52:44.699656 1391 raft_consensus.cc:493] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:52:44.699888 1391 raft_consensus.cc:515] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:44.700680 1391 leader_election.cc:290] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005)
I20260204 08:52:44.706374 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1e719e457e94a5da71929720411ae69" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d083ecc9895345e5b06cd00ef0246c71" is_pre_election: true
I20260204 08:52:44.737581 1214 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:52:44.737818 1214 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:44.738157 701 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1e719e457e94a5da71929720411ae69" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" is_pre_election: true
I20260204 08:52:44.738345 418 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu with pid 661
I20260204 08:52:44.738382 701 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1e719e457e94a5da71929720411ae69 in term 1.
I20260204 08:52:44.738555 1214 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:52:44.768918 817 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.104.190:42471 (error 108)
W20260204 08:52:44.780536 747 consensus_peers.cc:597] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 -> Peer ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471): Couldn't send request to peer ebeeb5d4823b41b3a8e37546ff2f6e34. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.190:42471: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260204 08:52:44.784653 1394 negotiation.cc:337] Failed RPC negotiation. Trace:
0204 08:52:44.770439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.104.190:42471 (local address 127.0.0.1:47732)
0204 08:52:44.783340 (+ 12901us) negotiation.cc:107] Waiting for socket to connect
0204 08:52:44.783531 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.104.190:42471: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":12721,"thread_start_us":12472,"threads_started":1}
W20260204 08:52:44.785034 817 leader_election.cc:336] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471): Network error: Client connection negotiation failed: client connection to 127.0.104.190:42471: connect: Connection reset by peer (error 104)
I20260204 08:52:44.785297 817 leader_election.cc:304] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1e719e457e94a5da71929720411ae69; no voters: d083ecc9895345e5b06cd00ef0246c71, ebeeb5d4823b41b3a8e37546ff2f6e34
I20260204 08:52:44.785684 1391 raft_consensus.cc:2749] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260204 08:52:44.910087 1067 heartbeater.cc:646] Failed to heartbeat to 127.0.104.190:42471 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.104.190:42471: connect: Connection refused (error 111)
W20260204 08:52:44.920446 1205 heartbeater.cc:646] Failed to heartbeat to 127.0.104.190:42471 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.104.190:42471: connect: Connection refused (error 111)
W20260204 08:52:44.963546 1341 heartbeater.cc:646] Failed to heartbeat to 127.0.104.190:42471 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.104.190:42471: connect: Connection refused (error 111)
I20260204 08:52:44.964449 1343 heartbeater.cc:499] Master 127.0.104.189:37005 was elected leader, sending a full tablet report...
I20260204 08:52:44.995399 1399 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:44.995890 1399 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: LEADER
I20260204 08:52:45.013614 762 catalog_manager.cc:2174] created HMS catalog entry for table [id=242a7b5626014adf9859874c46290421]
I20260204 08:52:45.046209 762 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41804:
table { table_name: "default.Table_ea715cabe3754310a06d2ce6559f0d4b" } new_table_name: "default.Table_67b2871860f446a1a5551b16cbbcec4d" modify_external_catalogs: true
I20260204 08:52:45.055881 761 catalog_manager.cc:2174] created HMS catalog entry for table [id=19930831c99540688fb9a1e6ebfd8f82]
I20260204 08:52:45.075176 761 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41804:
table { table_name: "default.Table_dd85a76e9cd94cc0b63f44ac7ed48a3a" } new_table_name: "default.Table_e7fe420574dc40cdb70c4602f5a6e0db" modify_external_catalogs: true
I20260204 08:52:45.098899 755 catalog_manager.cc:2174] created HMS catalog entry for table [id=893f125488f745a5b0bc9ec3d07da00b]
I20260204 08:52:45.110085 1276 tablet_service.cc:1505] Processing CreateTablet for tablet bcde541856bc450f84a15dbde23025f6 (DEFAULT_TABLE table=default.table_ea715cabe3754310a06d2ce6559f0d4b [id=242a7b5626014adf9859874c46290421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.111776 755 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41804:
table { table_name: "default.Table_798df4bc6a0640c4b26f8fe2b3444d6b" } new_table_name: "default.Table_9f93a995778f4fe6a4667cff0ef494b9" modify_external_catalogs: true
I20260204 08:52:45.111929 1276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcde541856bc450f84a15dbde23025f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.117034 1275 tablet_service.cc:1505] Processing CreateTablet for tablet 577191b33b244cdd9fc80f61fc28f39b (DEFAULT_TABLE table=default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.117594 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 577191b33b244cdd9fc80f61fc28f39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.119892 1274 tablet_service.cc:1505] Processing CreateTablet for tablet 4452a7c31cdc4144bc61a36f401dcde2 (DEFAULT_TABLE table=default.table_798df4bc6a0640c4b26f8fe2b3444d6b [id=893f125488f745a5b0bc9ec3d07da00b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.120476 1274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4452a7c31cdc4144bc61a36f401dcde2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.122370 1138 tablet_service.cc:1505] Processing CreateTablet for tablet 4452a7c31cdc4144bc61a36f401dcde2 (DEFAULT_TABLE table=default.table_798df4bc6a0640c4b26f8fe2b3444d6b [id=893f125488f745a5b0bc9ec3d07da00b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.121374 1139 tablet_service.cc:1505] Processing CreateTablet for tablet 577191b33b244cdd9fc80f61fc28f39b (DEFAULT_TABLE table=default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.123782 1140 tablet_service.cc:1505] Processing CreateTablet for tablet bcde541856bc450f84a15dbde23025f6 (DEFAULT_TABLE table=default.table_ea715cabe3754310a06d2ce6559f0d4b [id=242a7b5626014adf9859874c46290421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.123950 1138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4452a7c31cdc4144bc61a36f401dcde2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.125663 1140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcde541856bc450f84a15dbde23025f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.127159 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 577191b33b244cdd9fc80f61fc28f39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.130257 1414 tablet_bootstrap.cc:492] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:45.135473 1414 tablet_bootstrap.cc:654] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.137038 1414 log.cc:826] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134: Log is configured to *not* fsync() on all Append() calls
I20260204 08:52:45.139055 1001 tablet_service.cc:1505] Processing CreateTablet for tablet 577191b33b244cdd9fc80f61fc28f39b (DEFAULT_TABLE table=default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.140697 1000 tablet_service.cc:1505] Processing CreateTablet for tablet 4452a7c31cdc4144bc61a36f401dcde2 (DEFAULT_TABLE table=default.table_798df4bc6a0640c4b26f8fe2b3444d6b [id=893f125488f745a5b0bc9ec3d07da00b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.141476 1000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4452a7c31cdc4144bc61a36f401dcde2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.143740 1002 tablet_service.cc:1505] Processing CreateTablet for tablet bcde541856bc450f84a15dbde23025f6 (DEFAULT_TABLE table=default.table_ea715cabe3754310a06d2ce6559f0d4b [id=242a7b5626014adf9859874c46290421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.144321 1002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcde541856bc450f84a15dbde23025f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.153618 1001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 577191b33b244cdd9fc80f61fc28f39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.171499 1417 tablet_bootstrap.cc:492] T bcde541856bc450f84a15dbde23025f6 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:45.173662 1416 tablet_bootstrap.cc:492] T 577191b33b244cdd9fc80f61fc28f39b P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:45.176182 1417 tablet_bootstrap.cc:654] T bcde541856bc450f84a15dbde23025f6 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.177670 1417 log.cc:826] T bcde541856bc450f84a15dbde23025f6 P 6d471268b9e643e5bbbf8bde1644a5ae: Log is configured to *not* fsync() on all Append() calls
I20260204 08:52:45.187331 1417 tablet_bootstrap.cc:492] T bcde541856bc450f84a15dbde23025f6 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:45.187692 1417 ts_tablet_manager.cc:1403] T bcde541856bc450f84a15dbde23025f6 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260204 08:52:45.191407 1416 tablet_bootstrap.cc:654] T 577191b33b244cdd9fc80f61fc28f39b P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.192898 1416 log.cc:826] T 577191b33b244cdd9fc80f61fc28f39b P fdea56c2d1f5499087b939ec9f9dc070: Log is configured to *not* fsync() on all Append() calls
I20260204 08:52:45.194846 1417 raft_consensus.cc:359] T bcde541856bc450f84a15dbde23025f6 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.195304 1417 raft_consensus.cc:385] T bcde541856bc450f84a15dbde23025f6 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.195401 1417 raft_consensus.cc:740] T bcde541856bc450f84a15dbde23025f6 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.196180 1417 consensus_queue.cc:260] T bcde541856bc450f84a15dbde23025f6 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.199753 1414 tablet_bootstrap.cc:492] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:45.200127 1414 ts_tablet_manager.cc:1403] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.070s user 0.028s sys 0.036s
I20260204 08:52:45.201778 1417 ts_tablet_manager.cc:1434] T bcde541856bc450f84a15dbde23025f6 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
I20260204 08:52:45.207080 1414 raft_consensus.cc:359] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.207828 1414 raft_consensus.cc:385] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.207911 1417 tablet_bootstrap.cc:492] T 577191b33b244cdd9fc80f61fc28f39b P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:45.207953 1414 raft_consensus.cc:740] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.208709 1414 consensus_queue.cc:260] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.209880 1417 tablet_bootstrap.cc:654] T 577191b33b244cdd9fc80f61fc28f39b P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.210274 1416 tablet_bootstrap.cc:492] T 577191b33b244cdd9fc80f61fc28f39b P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:45.210656 1416 ts_tablet_manager.cc:1403] T 577191b33b244cdd9fc80f61fc28f39b P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.037s user 0.009s sys 0.000s
I20260204 08:52:45.211437 757 catalog_manager.cc:2174] created HMS catalog entry for table [id=2df47e721cd642e4b6fd5e61e172749c]
I20260204 08:52:45.217800 1416 raft_consensus.cc:359] T 577191b33b244cdd9fc80f61fc28f39b P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.218324 1416 raft_consensus.cc:385] T 577191b33b244cdd9fc80f61fc28f39b P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.218437 1416 raft_consensus.cc:740] T 577191b33b244cdd9fc80f61fc28f39b P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.219322 1416 consensus_queue.cc:260] T 577191b33b244cdd9fc80f61fc28f39b P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.229207 1416 ts_tablet_manager.cc:1434] T 577191b33b244cdd9fc80f61fc28f39b P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.018s user 0.011s sys 0.005s
I20260204 08:52:45.234097 1416 tablet_bootstrap.cc:492] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:45.234710 1139 tablet_service.cc:1505] Processing CreateTablet for tablet 7121dc10d79548169242f5b65331c08a (DEFAULT_TABLE table=default.table_42998faad7dd446c8a35b76a3a9b5994 [id=2df47e721cd642e4b6fd5e61e172749c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.234826 1002 tablet_service.cc:1505] Processing CreateTablet for tablet 7121dc10d79548169242f5b65331c08a (DEFAULT_TABLE table=default.table_42998faad7dd446c8a35b76a3a9b5994 [id=2df47e721cd642e4b6fd5e61e172749c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.235162 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7121dc10d79548169242f5b65331c08a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.235205 1002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7121dc10d79548169242f5b65331c08a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.236006 1416 tablet_bootstrap.cc:654] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.240218 1275 tablet_service.cc:1505] Processing CreateTablet for tablet 7121dc10d79548169242f5b65331c08a (DEFAULT_TABLE table=default.table_42998faad7dd446c8a35b76a3a9b5994 [id=2df47e721cd642e4b6fd5e61e172749c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.240705 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7121dc10d79548169242f5b65331c08a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.245883 757 master_service.cc:954] ReplaceTablet: received request to replace tablet bcde541856bc450f84a15dbde23025f6 from {username='slave'} at 127.0.0.1:41804
I20260204 08:52:45.249940 1416 tablet_bootstrap.cc:492] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:45.250140 1416 ts_tablet_manager.cc:1403] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.003s
I20260204 08:52:45.251153 1416 raft_consensus.cc:359] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.251377 1416 raft_consensus.cc:385] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.251487 1416 raft_consensus.cc:740] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.251770 1416 consensus_queue.cc:260] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.252439 1416 ts_tablet_manager.cc:1434] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260204 08:52:45.256114 1416 tablet_bootstrap.cc:492] T bcde541856bc450f84a15dbde23025f6 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:45.257683 757 catalog_manager.cc:5941] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Sending DeleteTablet for 3 replicas of tablet bcde541856bc450f84a15dbde23025f6
I20260204 08:52:45.258255 1416 tablet_bootstrap.cc:654] T bcde541856bc450f84a15dbde23025f6 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.259029 1275 tablet_service.cc:1552] Processing DeleteTablet for tablet bcde541856bc450f84a15dbde23025f6 with delete_type TABLET_DATA_DELETED (replaced by tablet cff3c1305f694fba837c0ced9cba71b0) from {username='slave'} at 127.0.0.1:38330
I20260204 08:52:45.259742 1002 tablet_service.cc:1552] Processing DeleteTablet for tablet bcde541856bc450f84a15dbde23025f6 with delete_type TABLET_DATA_DELETED (replaced by tablet cff3c1305f694fba837c0ced9cba71b0) from {username='slave'} at 127.0.0.1:39350
I20260204 08:52:45.261086 1139 tablet_service.cc:1552] Processing DeleteTablet for tablet bcde541856bc450f84a15dbde23025f6 with delete_type TABLET_DATA_DELETED (replaced by tablet cff3c1305f694fba837c0ced9cba71b0) from {username='slave'} at 127.0.0.1:52546
I20260204 08:52:45.266456 1416 tablet_bootstrap.cc:492] T bcde541856bc450f84a15dbde23025f6 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:45.266664 1416 ts_tablet_manager.cc:1403] T bcde541856bc450f84a15dbde23025f6 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260204 08:52:45.267689 1416 raft_consensus.cc:359] T bcde541856bc450f84a15dbde23025f6 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.267920 1416 raft_consensus.cc:385] T bcde541856bc450f84a15dbde23025f6 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.268009 1416 raft_consensus.cc:740] T bcde541856bc450f84a15dbde23025f6 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.268347 1416 consensus_queue.cc:260] T bcde541856bc450f84a15dbde23025f6 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.268882 1416 ts_tablet_manager.cc:1434] T bcde541856bc450f84a15dbde23025f6 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260204 08:52:45.269232 1416 tablet_bootstrap.cc:492] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:45.272969 1000 tablet_service.cc:1505] Processing CreateTablet for tablet cff3c1305f694fba837c0ced9cba71b0 (DEFAULT_TABLE table=default.table_ea715cabe3754310a06d2ce6559f0d4b [id=242a7b5626014adf9859874c46290421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.273515 1000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cff3c1305f694fba837c0ced9cba71b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.273782 1276 tablet_service.cc:1505] Processing CreateTablet for tablet cff3c1305f694fba837c0ced9cba71b0 (DEFAULT_TABLE table=default.table_ea715cabe3754310a06d2ce6559f0d4b [id=242a7b5626014adf9859874c46290421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.274230 1276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cff3c1305f694fba837c0ced9cba71b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.276429 1140 tablet_service.cc:1505] Processing CreateTablet for tablet cff3c1305f694fba837c0ced9cba71b0 (DEFAULT_TABLE table=default.table_ea715cabe3754310a06d2ce6559f0d4b [id=242a7b5626014adf9859874c46290421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.276906 1140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cff3c1305f694fba837c0ced9cba71b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.287472 757 catalog_manager.cc:6724] ReplaceTablet: tablet bcde541856bc450f84a15dbde23025f6 deleted and replaced by tablet cff3c1305f694fba837c0ced9cba71b0
I20260204 08:52:45.288707 757 master_service.cc:954] ReplaceTablet: received request to replace tablet 577191b33b244cdd9fc80f61fc28f39b from {username='slave'} at 127.0.0.1:41804
I20260204 08:52:45.289966 760 catalog_manager.cc:2174] created HMS catalog entry for table [id=105a9508f80c4190a40ab4ab66af2872]
I20260204 08:52:45.291426 1416 tablet_bootstrap.cc:654] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.296986 1428 tablet_replica.cc:333] T bcde541856bc450f84a15dbde23025f6 P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:52:45.297574 1428 raft_consensus.cc:2243] T bcde541856bc450f84a15dbde23025f6 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:45.298022 1428 raft_consensus.cc:2272] T bcde541856bc450f84a15dbde23025f6 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:45.303124 1416 tablet_bootstrap.cc:492] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:45.303426 1416 ts_tablet_manager.cc:1403] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.034s user 0.000s sys 0.004s
I20260204 08:52:45.303848 1428 ts_tablet_manager.cc:1916] T bcde541856bc450f84a15dbde23025f6 P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:45.304522 1416 raft_consensus.cc:359] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.304888 1416 raft_consensus.cc:385] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.305124 1416 raft_consensus.cc:740] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.305562 1416 consensus_queue.cc:260] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.306308 1416 ts_tablet_manager.cc:1434] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260204 08:52:45.306703 1416 tablet_bootstrap.cc:492] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:45.308801 1416 tablet_bootstrap.cc:654] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.310107 760 master_service.cc:954] ReplaceTablet: received request to replace tablet 577191b33b244cdd9fc80f61fc28f39b from {username='slave'} at 127.0.0.1:41804
W20260204 08:52:45.311995 749 catalog_manager.cc:4965] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): delete failed for tablet bcde541856bc450f84a15dbde23025f6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bcde541856bc450f84a15dbde23025f6 already in progress: creating tablet
I20260204 08:52:45.318187 760 master_service.cc:954] ReplaceTablet: received request to replace tablet 577191b33b244cdd9fc80f61fc28f39b from {username='slave'} at 127.0.0.1:41804
I20260204 08:52:45.319308 760 master_service.cc:954] ReplaceTablet: received request to replace tablet bcde541856bc450f84a15dbde23025f6 from {username='slave'} at 127.0.0.1:41804
I20260204 08:52:45.320461 760 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41804:
table { table_name: "default.Table_42998faad7dd446c8a35b76a3a9b5994" } new_table_name: "default.Table_f8273ef1a407421085918a8e3827f5f6" modify_external_catalogs: true
I20260204 08:52:45.321915 1428 ts_tablet_manager.cc:1929] T bcde541856bc450f84a15dbde23025f6 P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260204 08:52:45.322157 1428 log.cc:1199] T bcde541856bc450f84a15dbde23025f6 P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/bcde541856bc450f84a15dbde23025f6
I20260204 08:52:45.322777 1428 ts_tablet_manager.cc:1950] T bcde541856bc450f84a15dbde23025f6 P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:52:45.324204 1416 tablet_bootstrap.cc:492] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:45.324519 1416 ts_tablet_manager.cc:1403] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.000s
I20260204 08:52:45.325361 1416 raft_consensus.cc:359] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:45.325660 1416 raft_consensus.cc:385] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.325771 1416 raft_consensus.cc:740] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.326089 1416 consensus_queue.cc:260] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:45.326798 1416 ts_tablet_manager.cc:1434] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260204 08:52:45.332350 749 catalog_manager.cc:4985] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737): tablet bcde541856bc450f84a15dbde23025f6 (table default.table_ea715cabe3754310a06d2ce6559f0d4b [id=242a7b5626014adf9859874c46290421]) successfully deleted
I20260204 08:52:45.335645 1275 tablet_service.cc:1505] Processing CreateTablet for tablet 2de648ce42af4a2fbd494c8cc3e38bb4 (DEFAULT_TABLE table=default.table_3956d1827061406b8d0a913345864f74 [id=105a9508f80c4190a40ab4ab66af2872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.336249 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2de648ce42af4a2fbd494c8cc3e38bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.337095 1139 tablet_service.cc:1505] Processing CreateTablet for tablet 2de648ce42af4a2fbd494c8cc3e38bb4 (DEFAULT_TABLE table=default.table_3956d1827061406b8d0a913345864f74 [id=105a9508f80c4190a40ab4ab66af2872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.337185 1000 tablet_service.cc:1505] Processing CreateTablet for tablet 2de648ce42af4a2fbd494c8cc3e38bb4 (DEFAULT_TABLE table=default.table_3956d1827061406b8d0a913345864f74 [id=105a9508f80c4190a40ab4ab66af2872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.337549 1000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2de648ce42af4a2fbd494c8cc3e38bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.337570 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2de648ce42af4a2fbd494c8cc3e38bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.341145 757 catalog_manager.cc:5941] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Sending DeleteTablet for 3 replicas of tablet 577191b33b244cdd9fc80f61fc28f39b
I20260204 08:52:45.342130 1001 tablet_service.cc:1552] Processing DeleteTablet for tablet 577191b33b244cdd9fc80f61fc28f39b with delete_type TABLET_DATA_DELETED (replaced by tablet adcec26d4c104929b2880775a9cb29c5) from {username='slave'} at 127.0.0.1:39350
I20260204 08:52:45.343659 1427 tablet_replica.cc:333] T bcde541856bc450f84a15dbde23025f6 P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:52:45.344163 1416 tablet_bootstrap.cc:492] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:45.344197 1427 raft_consensus.cc:2243] T bcde541856bc450f84a15dbde23025f6 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:45.344672 1427 raft_consensus.cc:2272] T bcde541856bc450f84a15dbde23025f6 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:45.346010 1139 tablet_service.cc:1552] Processing DeleteTablet for tablet 577191b33b244cdd9fc80f61fc28f39b with delete_type TABLET_DATA_DELETED (replaced by tablet adcec26d4c104929b2880775a9cb29c5) from {username='slave'} at 127.0.0.1:52546
I20260204 08:52:45.346004 1416 tablet_bootstrap.cc:654] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.346308 1428 tablet_replica.cc:333] T 577191b33b244cdd9fc80f61fc28f39b P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:52:45.346534 1428 raft_consensus.cc:2243] T 577191b33b244cdd9fc80f61fc28f39b P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:45.346696 757 catalog_manager.cc:6724] ReplaceTablet: tablet 577191b33b244cdd9fc80f61fc28f39b deleted and replaced by tablet adcec26d4c104929b2880775a9cb29c5
I20260204 08:52:45.346747 1428 raft_consensus.cc:2272] T 577191b33b244cdd9fc80f61fc28f39b P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:45.346974 1275 tablet_service.cc:1552] Processing DeleteTablet for tablet 577191b33b244cdd9fc80f61fc28f39b with delete_type TABLET_DATA_DELETED (replaced by tablet adcec26d4c104929b2880775a9cb29c5) from {username='slave'} at 127.0.0.1:38330
W20260204 08:52:45.347726 749 catalog_manager.cc:4965] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): delete failed for tablet 577191b33b244cdd9fc80f61fc28f39b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 577191b33b244cdd9fc80f61fc28f39b already in progress: creating tablet
I20260204 08:52:45.348631 1428 ts_tablet_manager.cc:1916] T 577191b33b244cdd9fc80f61fc28f39b P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:45.354820 1428 ts_tablet_manager.cc:1929] T 577191b33b244cdd9fc80f61fc28f39b P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260204 08:52:45.355034 1428 log.cc:1199] T 577191b33b244cdd9fc80f61fc28f39b P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/577191b33b244cdd9fc80f61fc28f39b
I20260204 08:52:45.355445 1428 ts_tablet_manager.cc:1950] T 577191b33b244cdd9fc80f61fc28f39b P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:52:45.359720 757 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41804:
name: "default.Table_c97013dd16f84d8ba7cd528295f5ad49"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:45.361860 1427 ts_tablet_manager.cc:1916] T bcde541856bc450f84a15dbde23025f6 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:45.362825 1275 tablet_service.cc:1505] Processing CreateTablet for tablet adcec26d4c104929b2880775a9cb29c5 (DEFAULT_TABLE table=default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.363363 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adcec26d4c104929b2880775a9cb29c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.365531 1417 tablet_bootstrap.cc:492] T 577191b33b244cdd9fc80f61fc28f39b P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:45.365705 1417 ts_tablet_manager.cc:1403] T 577191b33b244cdd9fc80f61fc28f39b P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.158s user 0.003s sys 0.001s
I20260204 08:52:45.366707 1417 raft_consensus.cc:359] T 577191b33b244cdd9fc80f61fc28f39b P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.366943 1417 raft_consensus.cc:385] T 577191b33b244cdd9fc80f61fc28f39b P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.367054 1417 raft_consensus.cc:740] T 577191b33b244cdd9fc80f61fc28f39b P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.367412 1417 consensus_queue.cc:260] T 577191b33b244cdd9fc80f61fc28f39b P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.368706 1002 tablet_service.cc:1505] Processing CreateTablet for tablet adcec26d4c104929b2880775a9cb29c5 (DEFAULT_TABLE table=default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.369272 1002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adcec26d4c104929b2880775a9cb29c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.372284 1417 ts_tablet_manager.cc:1434] T 577191b33b244cdd9fc80f61fc28f39b P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260204 08:52:45.372781 1139 tablet_service.cc:1505] Processing CreateTablet for tablet adcec26d4c104929b2880775a9cb29c5 (DEFAULT_TABLE table=default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.373271 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adcec26d4c104929b2880775a9cb29c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.375599 1417 tablet_bootstrap.cc:492] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:45.377460 1417 tablet_bootstrap.cc:654] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.378608 759 catalog_manager.cc:2174] created HMS catalog entry for table [id=31385bebafdd42fe89d79858c81a5a18]
I20260204 08:52:45.389910 759 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41804:
name: "default.Table_732f1b588b344e49b2f77e17e6551853"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:45.391112 1416 tablet_bootstrap.cc:492] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:45.391327 1416 ts_tablet_manager.cc:1403] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.047s user 0.013s sys 0.018s
I20260204 08:52:45.392158 1416 raft_consensus.cc:359] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.392405 1416 raft_consensus.cc:385] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.392498 1416 raft_consensus.cc:740] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.392750 1416 consensus_queue.cc:260] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.394438 1416 ts_tablet_manager.cc:1434] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260204 08:52:45.395622 1416 tablet_bootstrap.cc:492] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:45.397827 1416 tablet_bootstrap.cc:654] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
W20260204 08:52:45.403975 1212 tablet.cc:2378] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260204 08:52:45.409340 749 catalog_manager.cc:4985] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737): tablet 577191b33b244cdd9fc80f61fc28f39b (table default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82]) successfully deleted
I20260204 08:52:45.411860 1275 tablet_service.cc:1505] Processing CreateTablet for tablet 7fdfc8c910a7462684ea737615bd2870 (DEFAULT_TABLE table=default.table_43ccd0dac1b24e20880a930959aa280a [id=31385bebafdd42fe89d79858c81a5a18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.412429 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fdfc8c910a7462684ea737615bd2870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.421769 1139 tablet_service.cc:1505] Processing CreateTablet for tablet 7fdfc8c910a7462684ea737615bd2870 (DEFAULT_TABLE table=default.table_43ccd0dac1b24e20880a930959aa280a [id=31385bebafdd42fe89d79858c81a5a18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.422214 1000 tablet_service.cc:1505] Processing CreateTablet for tablet 7fdfc8c910a7462684ea737615bd2870 (DEFAULT_TABLE table=default.table_43ccd0dac1b24e20880a930959aa280a [id=31385bebafdd42fe89d79858c81a5a18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.422464 1414 ts_tablet_manager.cc:1434] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.222s user 0.030s sys 0.039s
I20260204 08:52:45.422914 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fdfc8c910a7462684ea737615bd2870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.423199 1000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fdfc8c910a7462684ea737615bd2870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.425128 1414 tablet_bootstrap.cc:492] T bcde541856bc450f84a15dbde23025f6 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:45.427309 1414 tablet_bootstrap.cc:654] T bcde541856bc450f84a15dbde23025f6 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.429567 758 catalog_manager.cc:2174] created HMS catalog entry for table [id=da8943eba12e426ba3bac81a64141294]
I20260204 08:52:45.430925 1417 tablet_bootstrap.cc:492] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:45.431329 1417 ts_tablet_manager.cc:1403] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.056s user 0.005s sys 0.000s
I20260204 08:52:45.432277 1417 raft_consensus.cc:359] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.432628 1417 raft_consensus.cc:385] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.432770 1417 raft_consensus.cc:740] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.433143 1417 consensus_queue.cc:260] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.435546 1417 ts_tablet_manager.cc:1434] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260204 08:52:45.437065 1417 tablet_bootstrap.cc:492] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
W20260204 08:52:45.437233 1344 tablet.cc:2378] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260204 08:52:45.439364 1417 tablet_bootstrap.cc:654] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.440711 1427 ts_tablet_manager.cc:1929] T bcde541856bc450f84a15dbde23025f6 P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260204 08:52:45.440891 1427 log.cc:1199] T bcde541856bc450f84a15dbde23025f6 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/bcde541856bc450f84a15dbde23025f6
I20260204 08:52:45.441385 1427 ts_tablet_manager.cc:1950] T bcde541856bc450f84a15dbde23025f6 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:52:45.447968 1417 tablet_bootstrap.cc:492] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:45.448264 1417 ts_tablet_manager.cc:1403] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20260204 08:52:45.429549 1416 tablet_bootstrap.cc:492] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:45.448531 1416 ts_tablet_manager.cc:1403] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.053s user 0.000s sys 0.020s
I20260204 08:52:45.449430 1416 raft_consensus.cc:359] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.449663 1416 raft_consensus.cc:385] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.449739 1416 raft_consensus.cc:740] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.449977 1416 consensus_queue.cc:260] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.450496 1416 ts_tablet_manager.cc:1434] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260204 08:52:45.452279 1275 tablet_service.cc:1505] Processing CreateTablet for tablet 1a3763eb94ac42858c2986e3c11965f0 (DEFAULT_TABLE table=default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.452770 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a3763eb94ac42858c2986e3c11965f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.457419 1417 raft_consensus.cc:359] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.457782 1417 raft_consensus.cc:385] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.457945 1417 raft_consensus.cc:740] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.457865 758 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41804:
name: "default.Table_7bcbc3cfbc41406cba3ed79125a46e82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:45.458456 1417 consensus_queue.cc:260] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.462435 1002 tablet_service.cc:1505] Processing CreateTablet for tablet 1a3763eb94ac42858c2986e3c11965f0 (DEFAULT_TABLE table=default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.462741 1417 ts_tablet_manager.cc:1434] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.013s user 0.000s sys 0.002s
I20260204 08:52:45.462990 1002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a3763eb94ac42858c2986e3c11965f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.464866 1417 tablet_bootstrap.cc:492] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:45.466521 1427 tablet_replica.cc:333] T 577191b33b244cdd9fc80f61fc28f39b P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:52:45.466861 1427 raft_consensus.cc:2243] T 577191b33b244cdd9fc80f61fc28f39b P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:45.467082 1427 raft_consensus.cc:2272] T 577191b33b244cdd9fc80f61fc28f39b P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:45.468065 749 catalog_manager.cc:4985] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): tablet bcde541856bc450f84a15dbde23025f6 (table default.table_ea715cabe3754310a06d2ce6559f0d4b [id=242a7b5626014adf9859874c46290421]) successfully deleted
I20260204 08:52:45.471323 1414 tablet_bootstrap.cc:492] T bcde541856bc450f84a15dbde23025f6 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:45.471506 1414 ts_tablet_manager.cc:1403] T bcde541856bc450f84a15dbde23025f6 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.003s
I20260204 08:52:45.472435 1414 raft_consensus.cc:359] T bcde541856bc450f84a15dbde23025f6 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.472652 1414 raft_consensus.cc:385] T bcde541856bc450f84a15dbde23025f6 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.472736 1414 raft_consensus.cc:740] T bcde541856bc450f84a15dbde23025f6 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.473027 1414 consensus_queue.cc:260] T bcde541856bc450f84a15dbde23025f6 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.473248 1427 ts_tablet_manager.cc:1916] T 577191b33b244cdd9fc80f61fc28f39b P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:45.478157 1414 ts_tablet_manager.cc:1434] T bcde541856bc450f84a15dbde23025f6 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260204 08:52:45.481576 1417 tablet_bootstrap.cc:654] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.481633 1414 tablet_bootstrap.cc:492] T 577191b33b244cdd9fc80f61fc28f39b P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:45.482517 1140 tablet_service.cc:1505] Processing CreateTablet for tablet 1a3763eb94ac42858c2986e3c11965f0 (DEFAULT_TABLE table=default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.483031 1140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a3763eb94ac42858c2986e3c11965f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.483477 1414 tablet_bootstrap.cc:654] T 577191b33b244cdd9fc80f61fc28f39b P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.487983 754 catalog_manager.cc:2174] created HMS catalog entry for table [id=42a606fb50bf4c58be67f67afab90d53]
I20260204 08:52:45.489351 1416 tablet_bootstrap.cc:492] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:45.491201 1416 tablet_bootstrap.cc:654] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.493608 1414 tablet_bootstrap.cc:492] T 577191b33b244cdd9fc80f61fc28f39b P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:45.493816 1414 ts_tablet_manager.cc:1403] T 577191b33b244cdd9fc80f61fc28f39b P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260204 08:52:45.494619 1414 raft_consensus.cc:359] T 577191b33b244cdd9fc80f61fc28f39b P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.495225 1414 raft_consensus.cc:385] T 577191b33b244cdd9fc80f61fc28f39b P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.495347 1414 raft_consensus.cc:740] T 577191b33b244cdd9fc80f61fc28f39b P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.495627 1414 consensus_queue.cc:260] T 577191b33b244cdd9fc80f61fc28f39b P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.500075 1414 ts_tablet_manager.cc:1434] T 577191b33b244cdd9fc80f61fc28f39b P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260204 08:52:45.500831 754 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41804:
table { table_name: "default.Table_3956d1827061406b8d0a913345864f74" } new_table_name: "default.Table_f388b768c4b14290a90dd29962cc3fc4" modify_external_catalogs: true
I20260204 08:52:45.503455 1414 tablet_bootstrap.cc:492] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:45.504032 1427 ts_tablet_manager.cc:1929] T 577191b33b244cdd9fc80f61fc28f39b P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260204 08:52:45.504215 1427 log.cc:1199] T 577191b33b244cdd9fc80f61fc28f39b P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/577191b33b244cdd9fc80f61fc28f39b
I20260204 08:52:45.504593 1427 ts_tablet_manager.cc:1950] T 577191b33b244cdd9fc80f61fc28f39b P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:52:45.505304 1414 tablet_bootstrap.cc:654] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.515071 1275 tablet_service.cc:1552] Processing DeleteTablet for tablet bcde541856bc450f84a15dbde23025f6 with delete_type TABLET_DATA_DELETED (replaced by tablet cff3c1305f694fba837c0ced9cba71b0) from {username='slave'} at 127.0.0.1:38330
I20260204 08:52:45.515479 1426 tablet_replica.cc:333] T bcde541856bc450f84a15dbde23025f6 P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:52:45.515532 1276 tablet_service.cc:1505] Processing CreateTablet for tablet 81f37ac4f50a440d8e7292bfa05a0a7d (DEFAULT_TABLE table=default.table_73e876d60c444a5daa316e4bf7158614 [id=42a606fb50bf4c58be67f67afab90d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.516002 1276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81f37ac4f50a440d8e7292bfa05a0a7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.516011 1426 raft_consensus.cc:2243] T bcde541856bc450f84a15dbde23025f6 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:45.520826 1426 raft_consensus.cc:2272] T bcde541856bc450f84a15dbde23025f6 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:45.522521 1000 tablet_service.cc:1505] Processing CreateTablet for tablet 81f37ac4f50a440d8e7292bfa05a0a7d (DEFAULT_TABLE table=default.table_73e876d60c444a5daa316e4bf7158614 [id=42a606fb50bf4c58be67f67afab90d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.522984 1000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81f37ac4f50a440d8e7292bfa05a0a7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.523003 749 catalog_manager.cc:4985] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): tablet 577191b33b244cdd9fc80f61fc28f39b (table default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82]) successfully deleted
I20260204 08:52:45.523554 1426 ts_tablet_manager.cc:1916] T bcde541856bc450f84a15dbde23025f6 P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:45.523737 1414 tablet_bootstrap.cc:492] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:45.524578 1414 ts_tablet_manager.cc:1403] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.021s user 0.011s sys 0.002s
I20260204 08:52:45.525480 1414 raft_consensus.cc:359] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.525700 1414 raft_consensus.cc:385] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.525799 1414 raft_consensus.cc:740] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.526085 1414 consensus_queue.cc:260] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.529907 1414 ts_tablet_manager.cc:1434] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260204 08:52:45.530865 1139 tablet_service.cc:1505] Processing CreateTablet for tablet 81f37ac4f50a440d8e7292bfa05a0a7d (DEFAULT_TABLE table=default.table_73e876d60c444a5daa316e4bf7158614 [id=42a606fb50bf4c58be67f67afab90d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.531383 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81f37ac4f50a440d8e7292bfa05a0a7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.533515 1414 tablet_bootstrap.cc:492] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:45.533851 1426 ts_tablet_manager.cc:1929] T bcde541856bc450f84a15dbde23025f6 P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260204 08:52:45.534093 1426 log.cc:1199] T bcde541856bc450f84a15dbde23025f6 P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/bcde541856bc450f84a15dbde23025f6
I20260204 08:52:45.534691 1426 ts_tablet_manager.cc:1950] T bcde541856bc450f84a15dbde23025f6 P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:52:45.535377 1414 tablet_bootstrap.cc:654] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.540624 749 catalog_manager.cc:4985] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): tablet bcde541856bc450f84a15dbde23025f6 (table default.table_ea715cabe3754310a06d2ce6559f0d4b [id=242a7b5626014adf9859874c46290421]) successfully deleted
I20260204 08:52:45.546211 753 catalog_manager.cc:2174] created HMS catalog entry for table [id=c626dc1c029e4b59902b4493f4e16a78]
I20260204 08:52:45.553013 1416 tablet_bootstrap.cc:492] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:45.553146 1417 tablet_bootstrap.cc:492] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:45.553200 1416 ts_tablet_manager.cc:1403] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.064s user 0.004s sys 0.008s
I20260204 08:52:45.553319 1417 ts_tablet_manager.cc:1403] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.089s user 0.001s sys 0.011s
I20260204 08:52:45.553994 1417 raft_consensus.cc:359] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:45.553978 1416 raft_consensus.cc:359] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.554164 1416 raft_consensus.cc:385] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.554180 1417 raft_consensus.cc:385] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.554241 1417 raft_consensus.cc:740] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.554247 1416 raft_consensus.cc:740] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.554519 1417 consensus_queue.cc:260] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:45.555388 1416 consensus_queue.cc:260] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.557750 753 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41804 (request call id 10) took 1050 ms. Trace:
I20260204 08:52:45.557924 753 rpcz_store.cc:276] 0204 08:52:44.507184 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:44.507242 (+ 58us) service_pool.cc:225] Handling call
0204 08:52:44.520942 (+ 13700us) catalog_manager.cc:2101] Acquired catalog manager lock
0204 08:52:44.521412 (+ 470us) catalog_manager.cc:2147] Created new table and tablet info
0204 08:52:45.546152 (+1024740us) catalog_manager.cc:2173] Created new table in HMS catalog
0204 08:52:45.557454 (+ 11302us) catalog_manager.cc:2207] Wrote table and tablets to system table
0204 08:52:45.557593 (+ 139us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0204 08:52:45.557709 (+ 116us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:45.549586 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0204 08:52:45.549734 (+ 148us) write_op.cc:432] Acquiring schema lock in shared mode
0204 08:52:45.549742 (+ 8us) write_op.cc:435] Acquired schema lock
0204 08:52:45.549744 (+ 2us) tablet.cc:660] Decoding operations
0204 08:52:45.549846 (+ 102us) write_op.cc:620] Acquiring the partition lock for write op
0204 08:52:45.549861 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0204 08:52:45.549867 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0204 08:52:45.549960 (+ 93us) tablet.cc:699] Row locks acquired
0204 08:52:45.549962 (+ 2us) write_op.cc:260] PREPARE: finished
0204 08:52:45.550004 (+ 42us) write_op.cc:270] Start()
0204 08:52:45.550041 (+ 37us) write_op.cc:276] Timestamp: P: 1770195165549951 usec, L: 0
0204 08:52:45.550044 (+ 3us) op_driver.cc:348] REPLICATION: starting
0204 08:52:45.550254 (+ 210us) log.cc:844] Serialized 432 byte log entry
0204 08:52:45.556020 (+ 5766us) op_driver.cc:464] REPLICATION: finished
0204 08:52:45.556427 (+ 407us) write_op.cc:301] APPLY: starting
0204 08:52:45.556447 (+ 20us) tablet.cc:1365] starting BulkCheckPresence
0204 08:52:45.556495 (+ 48us) tablet.cc:1368] finished BulkCheckPresence
0204 08:52:45.556525 (+ 30us) tablet.cc:1370] starting ApplyRowOperation cycle
0204 08:52:45.556640 (+ 115us) tablet.cc:1381] finished ApplyRowOperation cycle
0204 08:52:45.556678 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0204 08:52:45.556685 (+ 7us) write_op.cc:312] APPLY: finished
0204 08:52:45.556777 (+ 92us) log.cc:844] Serialized 28 byte log entry
0204 08:52:45.556850 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:45.556876 (+ 26us) write_op.cc:454] Released schema lock
0204 08:52:45.556883 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0204 08:52:45.556961 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:45.556965 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":966344,"Hive Metastore.run_cpu_time_us":277,"Hive Metastore.run_wall_time_us":58245,"child_traces":[["op",{"apply.queue_time_us":338,"apply.run_cpu_time_us":529,"apply.run_wall_time_us":529,"num_ops":2,"prepare.queue_time_us":2579,"prepare.run_cpu_time_us":832,"prepare.run_wall_time_us":832,"raft.queue_time_us":1089,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":215,"replication_time_us":5941}]]}
I20260204 08:52:45.562548 1417 ts_tablet_manager.cc:1434] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260204 08:52:45.563154 1417 tablet_bootstrap.cc:492] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:45.564795 1423 raft_consensus.cc:493] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:45.565038 1423 raft_consensus.cc:515] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.567162 1423 leader_election.cc:290] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:45.569973 1276 tablet_service.cc:1552] Processing DeleteTablet for tablet bcde541856bc450f84a15dbde23025f6 with delete_type TABLET_DATA_DELETED (replaced by tablet cff3c1305f694fba837c0ced9cba71b0) from {username='slave'} at 127.0.0.1:38330
I20260204 08:52:45.570112 1275 tablet_service.cc:1552] Processing DeleteTablet for tablet 577191b33b244cdd9fc80f61fc28f39b with delete_type TABLET_DATA_DELETED (replaced by tablet adcec26d4c104929b2880775a9cb29c5) from {username='slave'} at 127.0.0.1:38330
I20260204 08:52:45.570508 1426 tablet_replica.cc:333] T 577191b33b244cdd9fc80f61fc28f39b P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:52:45.570716 1426 raft_consensus.cc:2243] T 577191b33b244cdd9fc80f61fc28f39b P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:45.570886 1426 raft_consensus.cc:2272] T 577191b33b244cdd9fc80f61fc28f39b P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:45.571985 1426 ts_tablet_manager.cc:1916] T 577191b33b244cdd9fc80f61fc28f39b P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260204 08:52:45.576928 749 catalog_manager.cc:4952] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): delete failed for tablet bcde541856bc450f84a15dbde23025f6 because the tablet was not found. No further retry: Not found: Tablet not found: bcde541856bc450f84a15dbde23025f6
I20260204 08:52:45.580804 756 catalog_manager.cc:2174] created HMS catalog entry for table [id=7d4edb505f7940668b492a9d592a3e61]
I20260204 08:52:45.584748 1275 tablet_service.cc:1505] Processing CreateTablet for tablet 80f72b647ee643afabcc5cafa9c43b13 (DEFAULT_TABLE table=default.table_9e950ede78084ef18f4e1051d9a42a90 [id=c626dc1c029e4b59902b4493f4e16a78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.585240 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80f72b647ee643afabcc5cafa9c43b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.585801 1416 ts_tablet_manager.cc:1434] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.032s user 0.000s sys 0.002s
I20260204 08:52:45.587375 1426 ts_tablet_manager.cc:1929] T 577191b33b244cdd9fc80f61fc28f39b P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260204 08:52:45.587543 1426 log.cc:1199] T 577191b33b244cdd9fc80f61fc28f39b P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/577191b33b244cdd9fc80f61fc28f39b
I20260204 08:52:45.587901 1426 ts_tablet_manager.cc:1950] T 577191b33b244cdd9fc80f61fc28f39b P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:52:45.589682 749 catalog_manager.cc:4985] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): tablet 577191b33b244cdd9fc80f61fc28f39b (table default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82]) successfully deleted
I20260204 08:52:45.592693 756 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41804 (request call id 7) took 1093 ms. Trace:
I20260204 08:52:45.592847 756 rpcz_store.cc:276] 0204 08:52:44.499452 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:44.499535 (+ 83us) service_pool.cc:225] Handling call
0204 08:52:44.521759 (+ 22224us) catalog_manager.cc:2101] Acquired catalog manager lock
0204 08:52:44.522168 (+ 409us) catalog_manager.cc:2147] Created new table and tablet info
0204 08:52:45.580747 (+1058579us) catalog_manager.cc:2173] Created new table in HMS catalog
0204 08:52:45.592407 (+ 11660us) catalog_manager.cc:2207] Wrote table and tablets to system table
0204 08:52:45.592527 (+ 120us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0204 08:52:45.592646 (+ 119us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:45.582697 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0204 08:52:45.582794 (+ 97us) write_op.cc:432] Acquiring schema lock in shared mode
0204 08:52:45.582800 (+ 6us) write_op.cc:435] Acquired schema lock
0204 08:52:45.582802 (+ 2us) tablet.cc:660] Decoding operations
0204 08:52:45.582897 (+ 95us) write_op.cc:620] Acquiring the partition lock for write op
0204 08:52:45.582910 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0204 08:52:45.582916 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0204 08:52:45.582994 (+ 78us) tablet.cc:699] Row locks acquired
0204 08:52:45.582996 (+ 2us) write_op.cc:260] PREPARE: finished
0204 08:52:45.583033 (+ 37us) write_op.cc:270] Start()
0204 08:52:45.583064 (+ 31us) write_op.cc:276] Timestamp: P: 1770195165582976 usec, L: 0
0204 08:52:45.583068 (+ 4us) op_driver.cc:348] REPLICATION: starting
0204 08:52:45.583247 (+ 179us) log.cc:844] Serialized 432 byte log entry
0204 08:52:45.591637 (+ 8390us) op_driver.cc:464] REPLICATION: finished
0204 08:52:45.591793 (+ 156us) write_op.cc:301] APPLY: starting
0204 08:52:45.591810 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
0204 08:52:45.591872 (+ 62us) tablet.cc:1368] finished BulkCheckPresence
0204 08:52:45.591905 (+ 33us) tablet.cc:1370] starting ApplyRowOperation cycle
0204 08:52:45.592042 (+ 137us) tablet.cc:1381] finished ApplyRowOperation cycle
0204 08:52:45.592075 (+ 33us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0204 08:52:45.592083 (+ 8us) write_op.cc:312] APPLY: finished
0204 08:52:45.592182 (+ 99us) log.cc:844] Serialized 28 byte log entry
0204 08:52:45.592253 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:45.592273 (+ 20us) write_op.cc:454] Released schema lock
0204 08:52:45.592280 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0204 08:52:45.592388 (+ 108us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:45.592393 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1023861,"Hive Metastore.run_cpu_time_us":309,"Hive Metastore.run_wall_time_us":34264,"child_traces":[["op",{"apply.queue_time_us":106,"apply.run_cpu_time_us":584,"apply.run_wall_time_us":583,"num_ops":2,"prepare.queue_time_us":1160,"prepare.run_cpu_time_us":668,"prepare.run_wall_time_us":669,"raft.queue_time_us":410,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":249,"replication_time_us":8542}]]}
I20260204 08:52:45.596912 1417 tablet_bootstrap.cc:654] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.598598 1416 tablet_bootstrap.cc:492] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:45.600499 1416 tablet_bootstrap.cc:654] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.602761 1000 tablet_service.cc:1505] Processing CreateTablet for tablet 80f72b647ee643afabcc5cafa9c43b13 (DEFAULT_TABLE table=default.table_9e950ede78084ef18f4e1051d9a42a90 [id=c626dc1c029e4b59902b4493f4e16a78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.603814 1000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80f72b647ee643afabcc5cafa9c43b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.608541 753 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41804:
table { table_name: "default.Table_43ccd0dac1b24e20880a930959aa280a" } modify_external_catalogs: true
I20260204 08:52:45.609105 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2de648ce42af4a2fbd494c8cc3e38bb4" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:45.609297 753 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41804:
table { table_name: "default.Table_43ccd0dac1b24e20880a930959aa280a" } modify_external_catalogs: true
I20260204 08:52:45.610374 1139 tablet_service.cc:1505] Processing CreateTablet for tablet 80f72b647ee643afabcc5cafa9c43b13 (DEFAULT_TABLE table=default.table_9e950ede78084ef18f4e1051d9a42a90 [id=c626dc1c029e4b59902b4493f4e16a78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.610926 1414 tablet_bootstrap.cc:492] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:45.610973 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80f72b647ee643afabcc5cafa9c43b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.611143 1414 ts_tablet_manager.cc:1403] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.078s user 0.001s sys 0.009s
W20260204 08:52:45.611436 1070 tablet.cc:2378] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260204 08:52:45.612016 1414 raft_consensus.cc:359] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:45.612265 1414 raft_consensus.cc:385] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.612382 1414 raft_consensus.cc:740] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.612648 1414 consensus_queue.cc:260] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:45.613163 1414 ts_tablet_manager.cc:1434] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260204 08:52:45.612979 756 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41804:
name: "default.Table_85f07b4907b24f1ca63791592455b59c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:45.614358 1002 tablet_service.cc:1505] Processing CreateTablet for tablet 5fe7a46289d84efcb7b1455bd317d279 (DEFAULT_TABLE table=default.table_b4244312363c4346b110598eb01428a3 [id=7d4edb505f7940668b492a9d592a3e61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.614576 1275 tablet_service.cc:1505] Processing CreateTablet for tablet 5fe7a46289d84efcb7b1455bd317d279 (DEFAULT_TABLE table=default.table_b4244312363c4346b110598eb01428a3 [id=7d4edb505f7940668b492a9d592a3e61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.614892 1002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fe7a46289d84efcb7b1455bd317d279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.614985 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fe7a46289d84efcb7b1455bd317d279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.617497 1416 tablet_bootstrap.cc:492] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:45.617766 1416 ts_tablet_manager.cc:1403] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.004s
I20260204 08:52:45.618605 1416 raft_consensus.cc:359] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.618906 1416 raft_consensus.cc:385] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.619033 1416 raft_consensus.cc:740] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.619484 1416 consensus_queue.cc:260] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.620168 1416 ts_tablet_manager.cc:1434] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260204 08:52:45.623368 1416 tablet_bootstrap.cc:492] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:45.625252 1416 tablet_bootstrap.cc:654] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.626576 1414 tablet_bootstrap.cc:492] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:45.628551 1414 tablet_bootstrap.cc:654] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.631084 1416 tablet_bootstrap.cc:492] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:45.631086 1417 tablet_bootstrap.cc:492] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:45.631265 1417 ts_tablet_manager.cc:1403] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.068s user 0.017s sys 0.008s
I20260204 08:52:45.631342 1416 ts_tablet_manager.cc:1403] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260204 08:52:45.632093 1416 raft_consensus.cc:359] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.632081 1417 raft_consensus.cc:359] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.632273 1416 raft_consensus.cc:385] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.632324 1417 raft_consensus.cc:385] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.632400 1416 raft_consensus.cc:740] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.632416 1417 raft_consensus.cc:740] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.632622 1417 consensus_queue.cc:260] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.632653 1416 consensus_queue.cc:260] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.632735 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2de648ce42af4a2fbd494c8cc3e38bb4" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:45.633833 1139 tablet_service.cc:1505] Processing CreateTablet for tablet 5fe7a46289d84efcb7b1455bd317d279 (DEFAULT_TABLE table=default.table_b4244312363c4346b110598eb01428a3 [id=7d4edb505f7940668b492a9d592a3e61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:45.633965 1417 ts_tablet_manager.cc:1434] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260204 08:52:45.634258 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fe7a46289d84efcb7b1455bd317d279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:45.634274 1417 tablet_bootstrap.cc:492] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:45.635973 1417 tablet_bootstrap.cc:654] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.636631 1416 ts_tablet_manager.cc:1434] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260204 08:52:45.637487 1421 raft_consensus.cc:493] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:45.637723 1421 raft_consensus.cc:515] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.639604 1421 leader_election.cc:290] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
W20260204 08:52:45.639899 1096 leader_election.cc:343] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Illegal state: must be running to vote when last-logged opid is not known
W20260204 08:52:45.640223 1096 leader_election.cc:343] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Illegal state: must be running to vote when last-logged opid is not known
I20260204 08:52:45.640390 1096 leader_election.cc:304] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:52:45.641108 1423 raft_consensus.cc:2749] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:52:45.642000 1423 raft_consensus.cc:493] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:45.642153 1423 raft_consensus.cc:515] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:45.642740 1423 leader_election.cc:290] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:45.644493 1414 tablet_bootstrap.cc:492] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:45.644645 1414 ts_tablet_manager.cc:1403] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.006s
I20260204 08:52:45.645414 1414 raft_consensus.cc:359] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.645599 1414 raft_consensus.cc:385] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.645682 1414 raft_consensus.cc:740] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.645881 1414 consensus_queue.cc:260] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.646381 1414 ts_tablet_manager.cc:1434] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260204 08:52:45.649291 1416 tablet_bootstrap.cc:492] T 80f72b647ee643afabcc5cafa9c43b13 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:45.651273 1416 tablet_bootstrap.cc:654] T 80f72b647ee643afabcc5cafa9c43b13 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.652086 1414 tablet_bootstrap.cc:492] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:45.653817 1414 tablet_bootstrap.cc:654] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.655303 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cff3c1305f694fba837c0ced9cba71b0" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:45.655339 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cff3c1305f694fba837c0ced9cba71b0" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:45.655584 1296 raft_consensus.cc:2468] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:45.655654 1022 raft_consensus.cc:2468] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:45.665170 1096 leader_election.cc:304] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:45.667032 1423 raft_consensus.cc:2804] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:45.667196 1423 raft_consensus.cc:493] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:45.667302 1423 raft_consensus.cc:3060] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.670390 1423 raft_consensus.cc:515] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:45.671146 1423 leader_election.cc:290] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Requested vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:45.671550 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4452a7c31cdc4144bc61a36f401dcde2" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:45.671792 1022 raft_consensus.cc:2468] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:45.672475 1232 leader_election.cc:304] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:45.672894 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cff3c1305f694fba837c0ced9cba71b0" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:45.673151 1296 raft_consensus.cc:3060] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.674283 1414 tablet_bootstrap.cc:492] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:45.674467 1414 ts_tablet_manager.cc:1403] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.000s
I20260204 08:52:45.675347 1414 raft_consensus.cc:359] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.675530 1414 raft_consensus.cc:385] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.675595 1414 raft_consensus.cc:740] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.675856 1414 consensus_queue.cc:260] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.676373 1414 ts_tablet_manager.cc:1434] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260204 08:52:45.676725 1414 tablet_bootstrap.cc:492] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:45.677826 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cff3c1305f694fba837c0ced9cba71b0" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:45.678077 1022 raft_consensus.cc:3060] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.678511 1414 tablet_bootstrap.cc:654] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.682030 1022 raft_consensus.cc:2468] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:45.685050 1096 leader_election.cc:304] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:45.685421 1423 raft_consensus.cc:2804] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:45.676247 1296 raft_consensus.cc:2468] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:45.683501 1417 tablet_bootstrap.cc:492] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:45.692795 1417 ts_tablet_manager.cc:1403] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.059s user 0.017s sys 0.030s
I20260204 08:52:45.693998 1421 raft_consensus.cc:2804] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:45.694164 1421 raft_consensus.cc:493] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:45.694247 1421 raft_consensus.cc:3060] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.694239 1423 raft_consensus.cc:697] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Becoming Leader. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Running, Role: LEADER
I20260204 08:52:45.694762 1423 consensus_queue.cc:237] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:45.697335 1421 raft_consensus.cc:515] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.697938 1417 raft_consensus.cc:359] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.698199 1417 raft_consensus.cc:385] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.698279 1417 raft_consensus.cc:740] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.698112 1421 leader_election.cc:290] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:45.698526 1417 consensus_queue.cc:260] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.699210 1417 ts_tablet_manager.cc:1434] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260204 08:52:45.700191 1417 tablet_bootstrap.cc:492] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:45.702193 1417 tablet_bootstrap.cc:654] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.704438 1416 tablet_bootstrap.cc:492] T 80f72b647ee643afabcc5cafa9c43b13 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:45.704655 1416 ts_tablet_manager.cc:1403] T 80f72b647ee643afabcc5cafa9c43b13 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.055s user 0.003s sys 0.023s
I20260204 08:52:45.704761 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4452a7c31cdc4144bc61a36f401dcde2" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:45.705457 1414 tablet_bootstrap.cc:492] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:45.705623 1414 ts_tablet_manager.cc:1403] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.003s
I20260204 08:52:45.705782 1416 raft_consensus.cc:359] T 80f72b647ee643afabcc5cafa9c43b13 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.706029 1416 raft_consensus.cc:385] T 80f72b647ee643afabcc5cafa9c43b13 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.706105 1416 raft_consensus.cc:740] T 80f72b647ee643afabcc5cafa9c43b13 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.706415 1414 raft_consensus.cc:359] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.706426 1416 consensus_queue.cc:260] T 80f72b647ee643afabcc5cafa9c43b13 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.706600 1414 raft_consensus.cc:385] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.706115 1022 raft_consensus.cc:3060] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.706686 1414 raft_consensus.cc:740] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.707844 1414 consensus_queue.cc:260] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.708213 1416 ts_tablet_manager.cc:1434] T 80f72b647ee643afabcc5cafa9c43b13 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260204 08:52:45.709551 1416 tablet_bootstrap.cc:492] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:45.710970 1414 ts_tablet_manager.cc:1434] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260204 08:52:45.712402 1416 tablet_bootstrap.cc:654] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.714010 1022 raft_consensus.cc:2468] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:45.715065 1232 leader_election.cc:304] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:45.716118 1160 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4452a7c31cdc4144bc61a36f401dcde2" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:45.716594 1160 raft_consensus.cc:2468] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:45.716143 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4452a7c31cdc4144bc61a36f401dcde2" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:45.717290 1159 raft_consensus.cc:3060] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.720579 1159 raft_consensus.cc:2468] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:45.721099 1421 raft_consensus.cc:2804] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:45.722244 1453 raft_consensus.cc:493] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:45.722455 1453 raft_consensus.cc:515] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.723378 1453 leader_election.cc:290] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:45.724257 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "adcec26d4c104929b2880775a9cb29c5" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:45.724539 1296 raft_consensus.cc:2468] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:45.725829 1096 leader_election.cc:304] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:45.726289 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "adcec26d4c104929b2880775a9cb29c5" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:45.726645 1022 raft_consensus.cc:2468] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:45.727551 1421 raft_consensus.cc:697] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Becoming Leader. State: Replica: 1234acd7d0db405085a5d95795007134, State: Running, Role: LEADER
I20260204 08:52:45.728047 1421 consensus_queue.cc:237] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.730468 1423 raft_consensus.cc:2804] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:45.730630 1423 raft_consensus.cc:493] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:45.730720 1423 raft_consensus.cc:3060] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.730756 1414 tablet_bootstrap.cc:492] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:45.732573 1414 tablet_bootstrap.cc:654] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.739023 1423 raft_consensus.cc:515] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.739807 1423 leader_election.cc:290] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Requested vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:45.740597 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "adcec26d4c104929b2880775a9cb29c5" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:45.740872 1022 raft_consensus.cc:3060] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.742846 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "adcec26d4c104929b2880775a9cb29c5" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:45.743101 1296 raft_consensus.cc:3060] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.743285 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
/tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:42471
--webserver_interface=127.0.104.190
--webserver_port=40787
--builtin_ntp_servers=127.0.104.148:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260204 08:52:45.746446 1296 raft_consensus.cc:2468] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:45.747190 1096 leader_election.cc:304] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:45.747619 1423 raft_consensus.cc:2804] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:45.749539 1423 raft_consensus.cc:697] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Becoming Leader. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Running, Role: LEADER
I20260204 08:52:45.749958 1423 consensus_queue.cc:237] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.750365 1022 raft_consensus.cc:2468] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:45.751977 1417 tablet_bootstrap.cc:492] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:45.756735 1417 ts_tablet_manager.cc:1403] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.057s user 0.019s sys 0.015s
I20260204 08:52:45.761322 1417 raft_consensus.cc:359] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.763770 1417 raft_consensus.cc:385] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.764344 1417 raft_consensus.cc:740] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.764763 1417 consensus_queue.cc:260] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.765890 1417 ts_tablet_manager.cc:1434] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260204 08:52:45.766101 1416 tablet_bootstrap.cc:492] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:45.766263 1416 ts_tablet_manager.cc:1403] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.057s user 0.009s sys 0.020s
I20260204 08:52:45.766350 1414 tablet_bootstrap.cc:492] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:45.766523 1414 ts_tablet_manager.cc:1403] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.036s user 0.007s sys 0.008s
I20260204 08:52:45.767027 1416 raft_consensus.cc:359] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.767233 1416 raft_consensus.cc:385] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.767316 1416 raft_consensus.cc:740] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.767264 1414 raft_consensus.cc:359] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.767463 1414 raft_consensus.cc:385] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.767530 1414 raft_consensus.cc:740] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.767794 1414 consensus_queue.cc:260] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.768450 1416 consensus_queue.cc:260] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.768960 1416 ts_tablet_manager.cc:1434] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260204 08:52:45.769111 1414 ts_tablet_manager.cc:1434] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260204 08:52:45.769569 1417 tablet_bootstrap.cc:492] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:45.770196 1414 tablet_bootstrap.cc:492] T 81f37ac4f50a440d8e7292bfa05a0a7d P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:45.771924 1414 tablet_bootstrap.cc:654] T 81f37ac4f50a440d8e7292bfa05a0a7d P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.771935 1417 tablet_bootstrap.cc:654] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.779080 1414 tablet_bootstrap.cc:492] T 81f37ac4f50a440d8e7292bfa05a0a7d P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:45.779268 1414 ts_tablet_manager.cc:1403] T 81f37ac4f50a440d8e7292bfa05a0a7d P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260204 08:52:45.780267 1414 raft_consensus.cc:359] T 81f37ac4f50a440d8e7292bfa05a0a7d P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.780511 1414 raft_consensus.cc:385] T 81f37ac4f50a440d8e7292bfa05a0a7d P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.780599 1414 raft_consensus.cc:740] T 81f37ac4f50a440d8e7292bfa05a0a7d P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.780894 1414 consensus_queue.cc:260] T 81f37ac4f50a440d8e7292bfa05a0a7d P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.785147 1414 ts_tablet_manager.cc:1434] T 81f37ac4f50a440d8e7292bfa05a0a7d P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260204 08:52:45.788942 1414 tablet_bootstrap.cc:492] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:45.791108 1414 tablet_bootstrap.cc:654] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.792188 1417 tablet_bootstrap.cc:492] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:45.792423 1417 ts_tablet_manager.cc:1403] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.002s
I20260204 08:52:45.793182 1417 raft_consensus.cc:359] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.793548 1417 raft_consensus.cc:385] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.793632 1417 raft_consensus.cc:740] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.793890 1417 consensus_queue.cc:260] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.794423 1417 ts_tablet_manager.cc:1434] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260204 08:52:45.794759 1417 tablet_bootstrap.cc:492] T 81f37ac4f50a440d8e7292bfa05a0a7d P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:45.796782 1417 tablet_bootstrap.cc:654] T 81f37ac4f50a440d8e7292bfa05a0a7d P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.799685 1414 tablet_bootstrap.cc:492] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:45.799897 1414 ts_tablet_manager.cc:1403] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.009s
I20260204 08:52:45.800776 1414 raft_consensus.cc:359] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.801028 1414 raft_consensus.cc:385] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.801122 1414 raft_consensus.cc:740] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.801450 1414 consensus_queue.cc:260] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.805868 1414 ts_tablet_manager.cc:1434] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260204 08:52:45.806565 1414 tablet_bootstrap.cc:492] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:45.808357 1414 tablet_bootstrap.cc:654] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.810696 1414 tablet_bootstrap.cc:492] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:45.810835 1414 ts_tablet_manager.cc:1403] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260204 08:52:45.811630 1414 raft_consensus.cc:359] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.811856 1414 raft_consensus.cc:385] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.811957 1414 raft_consensus.cc:740] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.812284 1414 consensus_queue.cc:260] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.813010 1414 ts_tablet_manager.cc:1434] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260204 08:52:45.815173 1417 tablet_bootstrap.cc:492] T 81f37ac4f50a440d8e7292bfa05a0a7d P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:45.815353 1417 ts_tablet_manager.cc:1403] T 81f37ac4f50a440d8e7292bfa05a0a7d P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.005s
I20260204 08:52:45.816237 1417 raft_consensus.cc:359] T 81f37ac4f50a440d8e7292bfa05a0a7d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.816468 1417 raft_consensus.cc:385] T 81f37ac4f50a440d8e7292bfa05a0a7d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.816566 1417 raft_consensus.cc:740] T 81f37ac4f50a440d8e7292bfa05a0a7d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.816805 1417 consensus_queue.cc:260] T 81f37ac4f50a440d8e7292bfa05a0a7d P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.817327 1417 ts_tablet_manager.cc:1434] T 81f37ac4f50a440d8e7292bfa05a0a7d P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260204 08:52:45.817632 1417 tablet_bootstrap.cc:492] T 80f72b647ee643afabcc5cafa9c43b13 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:45.819497 1417 tablet_bootstrap.cc:654] T 80f72b647ee643afabcc5cafa9c43b13 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.831449 1421 raft_consensus.cc:493] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:45.831699 1421 raft_consensus.cc:515] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.832516 1421 leader_election.cc:290] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:45.833845 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7121dc10d79548169242f5b65331c08a" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:45.834141 1022 raft_consensus.cc:2468] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:45.834553 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7121dc10d79548169242f5b65331c08a" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:45.834798 1159 raft_consensus.cc:2468] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:45.835593 1232 leader_election.cc:304] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:45.836215 1421 raft_consensus.cc:2804] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:45.836383 1421 raft_consensus.cc:493] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:45.836498 1421 raft_consensus.cc:3060] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.839201 1421 raft_consensus.cc:515] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.840665 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7121dc10d79548169242f5b65331c08a" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:45.840894 1022 raft_consensus.cc:3060] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.842900 1423 raft_consensus.cc:493] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:45.843106 1423 raft_consensus.cc:515] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.843755 1423 leader_election.cc:290] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:45.843766 1022 raft_consensus.cc:2468] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:45.844657 1232 leader_election.cc:304] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:45.844933 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a3763eb94ac42858c2986e3c11965f0" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:45.845026 1457 raft_consensus.cc:2804] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:45.844903 1021 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a3763eb94ac42858c2986e3c11965f0" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:45.845163 1296 raft_consensus.cc:2468] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:45.845237 1021 raft_consensus.cc:2468] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:45.846134 1096 leader_election.cc:304] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:45.847036 1423 raft_consensus.cc:2804] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:45.847148 1423 raft_consensus.cc:493] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:45.847254 1423 raft_consensus.cc:3060] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.847790 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7121dc10d79548169242f5b65331c08a" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:45.848040 1159 raft_consensus.cc:3060] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.850977 1423 raft_consensus.cc:515] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.851445 1159 raft_consensus.cc:2468] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:45.852406 1421 leader_election.cc:290] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:45.852492 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a3763eb94ac42858c2986e3c11965f0" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:45.852721 1296 raft_consensus.cc:3060] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.855916 1296 raft_consensus.cc:2468] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:45.856564 1096 leader_election.cc:304] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:45.856927 1453 raft_consensus.cc:2804] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:45.860646 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a3763eb94ac42858c2986e3c11965f0" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:45.861022 1022 raft_consensus.cc:3060] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.862187 1457 raft_consensus.cc:697] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Becoming Leader. State: Replica: 1234acd7d0db405085a5d95795007134, State: Running, Role: LEADER
I20260204 08:52:45.862701 1457 consensus_queue.cc:237] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:45.863270 1423 leader_election.cc:290] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Requested vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:45.867911 1417 tablet_bootstrap.cc:492] T 80f72b647ee643afabcc5cafa9c43b13 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:45.868136 1417 ts_tablet_manager.cc:1403] T 80f72b647ee643afabcc5cafa9c43b13 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.051s user 0.010s sys 0.004s
I20260204 08:52:45.868633 1022 raft_consensus.cc:2468] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:45.869339 1417 raft_consensus.cc:359] T 80f72b647ee643afabcc5cafa9c43b13 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.869551 1417 raft_consensus.cc:385] T 80f72b647ee643afabcc5cafa9c43b13 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.869621 1417 raft_consensus.cc:740] T 80f72b647ee643afabcc5cafa9c43b13 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.869877 1417 consensus_queue.cc:260] T 80f72b647ee643afabcc5cafa9c43b13 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.874011 1417 ts_tablet_manager.cc:1434] T 80f72b647ee643afabcc5cafa9c43b13 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260204 08:52:45.876392 1417 tablet_bootstrap.cc:492] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:45.878835 1417 tablet_bootstrap.cc:654] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:45.883916 1453 raft_consensus.cc:697] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Becoming Leader. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Running, Role: LEADER
I20260204 08:52:45.884337 1453 consensus_queue.cc:237] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.889678 1417 tablet_bootstrap.cc:492] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:45.889967 1417 ts_tablet_manager.cc:1403] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.000s
I20260204 08:52:45.890898 1417 raft_consensus.cc:359] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.891211 1417 raft_consensus.cc:385] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:45.891363 1417 raft_consensus.cc:740] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:45.891832 1417 consensus_queue.cc:260] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.892567 1417 ts_tablet_manager.cc:1434] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260204 08:52:45.949195 1457 raft_consensus.cc:493] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:45.949451 1457 raft_consensus.cc:515] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.950238 1457 leader_election.cc:290] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:45.950968 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80f72b647ee643afabcc5cafa9c43b13" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:45.951217 1022 raft_consensus.cc:2468] T 80f72b647ee643afabcc5cafa9c43b13 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:45.951226 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80f72b647ee643afabcc5cafa9c43b13" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:45.951458 1159 raft_consensus.cc:2468] T 80f72b647ee643afabcc5cafa9c43b13 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:45.952216 1232 leader_election.cc:304] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:45.952764 1457 raft_consensus.cc:2804] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:45.952850 1457 raft_consensus.cc:493] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:45.952934 1457 raft_consensus.cc:3060] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.955693 1457 raft_consensus.cc:515] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.957372 1457 leader_election.cc:290] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:45.957356 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80f72b647ee643afabcc5cafa9c43b13" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:45.957556 1022 raft_consensus.cc:3060] T 80f72b647ee643afabcc5cafa9c43b13 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.958221 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80f72b647ee643afabcc5cafa9c43b13" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:45.958434 1159 raft_consensus.cc:3060] T 80f72b647ee643afabcc5cafa9c43b13 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.960283 1022 raft_consensus.cc:2468] T 80f72b647ee643afabcc5cafa9c43b13 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:45.960983 1159 raft_consensus.cc:2468] T 80f72b647ee643afabcc5cafa9c43b13 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:45.961766 1232 leader_election.cc:304] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:45.962451 1457 raft_consensus.cc:2804] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:45.962662 1457 raft_consensus.cc:697] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Becoming Leader. State: Replica: 1234acd7d0db405085a5d95795007134, State: Running, Role: LEADER
I20260204 08:52:45.962970 1457 consensus_queue.cc:237] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.975931 1453 raft_consensus.cc:493] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:45.976169 1453 raft_consensus.cc:515] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.976999 1453 leader_election.cc:290] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:45.978302 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "81f37ac4f50a440d8e7292bfa05a0a7d" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:45.978566 1296 raft_consensus.cc:2468] T 81f37ac4f50a440d8e7292bfa05a0a7d P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:45.979000 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "81f37ac4f50a440d8e7292bfa05a0a7d" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:45.979240 1022 raft_consensus.cc:2468] T 81f37ac4f50a440d8e7292bfa05a0a7d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:45.979746 1096 leader_election.cc:304] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:45.980504 1453 raft_consensus.cc:2804] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:45.980614 1453 raft_consensus.cc:493] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:45.980700 1453 raft_consensus.cc:3060] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.983423 1453 raft_consensus.cc:515] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.984627 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "81f37ac4f50a440d8e7292bfa05a0a7d" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:45.984835 1296 raft_consensus.cc:3060] T 81f37ac4f50a440d8e7292bfa05a0a7d P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.987493 1296 raft_consensus.cc:2468] T 81f37ac4f50a440d8e7292bfa05a0a7d P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:45.988039 1096 leader_election.cc:304] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:45.988418 1469 raft_consensus.cc:2804] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:45.988559 1469 raft_consensus.cc:697] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Becoming Leader. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Running, Role: LEADER
I20260204 08:52:45.988829 1469 consensus_queue.cc:237] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:45.992455 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "81f37ac4f50a440d8e7292bfa05a0a7d" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:45.992722 1022 raft_consensus.cc:3060] T 81f37ac4f50a440d8e7292bfa05a0a7d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:45.995419 1022 raft_consensus.cc:2468] T 81f37ac4f50a440d8e7292bfa05a0a7d P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:45.996178 1453 leader_election.cc:290] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Requested vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:46.005596 1457 raft_consensus.cc:493] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:46.005787 1457 raft_consensus.cc:515] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:46.006460 1457 leader_election.cc:290] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:46.007495 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2de648ce42af4a2fbd494c8cc3e38bb4" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:46.007606 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2de648ce42af4a2fbd494c8cc3e38bb4" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:46.007723 1022 raft_consensus.cc:2468] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:46.007829 1159 raft_consensus.cc:2468] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:46.008370 1232 leader_election.cc:304] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:46.008948 1457 raft_consensus.cc:2804] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:46.009091 1457 raft_consensus.cc:493] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:46.009188 1457 raft_consensus.cc:3060] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:46.011976 1457 raft_consensus.cc:515] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:46.013573 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2de648ce42af4a2fbd494c8cc3e38bb4" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:46.013554 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2de648ce42af4a2fbd494c8cc3e38bb4" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:46.013782 1022 raft_consensus.cc:3060] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:46.013809 1159 raft_consensus.cc:3060] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:46.016566 1159 raft_consensus.cc:2468] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:46.016582 1022 raft_consensus.cc:2468] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:46.016903 1457 leader_election.cc:290] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:46.017419 1232 leader_election.cc:304] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:46.017746 1457 raft_consensus.cc:2804] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:46.019070 1457 raft_consensus.cc:697] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Becoming Leader. State: Replica: 1234acd7d0db405085a5d95795007134, State: Running, Role: LEADER
I20260204 08:52:46.019568 1457 consensus_queue.cc:237] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:46.088346 1469 raft_consensus.cc:493] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:46.088594 1469 raft_consensus.cc:515] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:46.089414 1469 leader_election.cc:290] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:46.090929 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5fe7a46289d84efcb7b1455bd317d279" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:46.091212 1022 raft_consensus.cc:2468] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:46.091575 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5fe7a46289d84efcb7b1455bd317d279" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:46.091809 1296 raft_consensus.cc:2468] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:46.092630 1096 leader_election.cc:304] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:46.093291 1469 raft_consensus.cc:2804] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:46.093387 1469 raft_consensus.cc:493] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:46.093478 1469 raft_consensus.cc:3060] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:46.096326 1469 raft_consensus.cc:515] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:46.097641 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5fe7a46289d84efcb7b1455bd317d279" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:46.097890 1022 raft_consensus.cc:3060] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:46.101115 1022 raft_consensus.cc:2468] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:46.101807 1096 leader_election.cc:304] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:46.102231 1453 raft_consensus.cc:2804] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:46.102363 1453 raft_consensus.cc:697] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Becoming Leader. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Running, Role: LEADER
I20260204 08:52:46.102638 1453 consensus_queue.cc:237] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:46.107517 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5fe7a46289d84efcb7b1455bd317d279" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:46.107776 1296 raft_consensus.cc:3060] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:46.110570 1296 raft_consensus.cc:2468] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:46.114811 1469 leader_election.cc:290] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:46.122237 1477 raft_consensus.cc:493] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:52:46.122606 1477 raft_consensus.cc:515] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:46.127605 1477 leader_election.cc:290] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005)
I20260204 08:52:46.127600 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1e719e457e94a5da71929720411ae69" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: false dest_uuid: "d083ecc9895345e5b06cd00ef0246c71" is_pre_election: true
I20260204 08:52:46.132221 762 catalog_manager.cc:3485] renamed table default.table_ea715cabe3754310a06d2ce6559f0d4b [id=242a7b5626014adf9859874c46290421] in HMS: new name default.table_67b2871860f446a1a5551b16cbbcec4d
W20260204 08:52:46.137168 757 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_c97013dd16f84d8ba7cd528295f5ad49 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260204 08:52:46.137197 759 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_732f1b588b344e49b2f77e17e6551853 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260204 08:52:46.138587 758 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_7bcbc3cfbc41406cba3ed79125a46e82 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260204 08:52:46.143976 817 leader_election.cc:336] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471): Network error: Client connection negotiation failed: client connection to 127.0.104.190:42471: connect: Connection refused (error 111)
I20260204 08:52:46.144153 817 leader_election.cc:304] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1e719e457e94a5da71929720411ae69; no voters: d083ecc9895345e5b06cd00ef0246c71, ebeeb5d4823b41b3a8e37546ff2f6e34
I20260204 08:52:46.144598 1477 raft_consensus.cc:2749] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260204 08:52:46.200700 924 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cff3c1305f694fba837c0ced9cba71b0 consensus state has no leader
W20260204 08:52:46.203148 756 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_85f07b4907b24f1ca63791592455b59c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260204 08:52:46.204804 762 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:41804 (request call id 11) took 1159 ms. Trace:
I20260204 08:52:46.204965 762 rpcz_store.cc:276] 0204 08:52:45.045481 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:45.045541 (+ 60us) service_pool.cc:225] Handling call
0204 08:52:45.596410 (+550869us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0204 08:52:46.204054 (+607644us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0204 08:52:46.204435 (+ 381us) catalog_manager.cc:3708] Apply alter schema
0204 08:52:46.204753 (+ 318us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":4973,"Hive Metastore.run_cpu_time_us":1149,"Hive Metastore.run_wall_time_us":530486}
I20260204 08:52:46.206089 762 catalog_manager.cc:5746] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491) does not have the latest schema for tablet cff3c1305f694fba837c0ced9cba71b0 (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
I20260204 08:52:46.215975 762 catalog_manager.cc:5654] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1234acd7d0db405085a5d95795007134 (127.0.104.131). New cstate: current_term: 1 leader_uuid: "1234acd7d0db405085a5d95795007134" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: UNKNOWN } } }
I20260204 08:52:46.217128 762 catalog_manager.cc:5654] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1234acd7d0db405085a5d95795007134 (127.0.104.131). New cstate: current_term: 1 leader_uuid: "1234acd7d0db405085a5d95795007134" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: HEALTHY } } }
I20260204 08:52:46.217737 762 catalog_manager.cc:5654] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1234acd7d0db405085a5d95795007134 (127.0.104.131). New cstate: current_term: 1 leader_uuid: "1234acd7d0db405085a5d95795007134" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: UNKNOWN } } }
I20260204 08:52:46.218273 762 catalog_manager.cc:5654] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1234acd7d0db405085a5d95795007134 (127.0.104.131). New cstate: current_term: 1 leader_uuid: "1234acd7d0db405085a5d95795007134" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: HEALTHY } } }
I20260204 08:52:46.233506 760 catalog_manager.cc:3485] renamed table default.table_42998faad7dd446c8a35b76a3a9b5994 [id=2df47e721cd642e4b6fd5e61e172749c] in HMS: new name default.table_f8273ef1a407421085918a8e3827f5f6
I20260204 08:52:46.242795 1457 raft_consensus.cc:493] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:46.243036 1457 raft_consensus.cc:515] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:46.242975 762 catalog_manager.cc:5654] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 reported cstate change: term changed from 0 to 1, leader changed from <none> to fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "fdea56c2d1f5499087b939ec9f9dc070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: HEALTHY } } }
I20260204 08:52:46.243571 762 catalog_manager.cc:5654] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 reported cstate change: term changed from 0 to 1, leader changed from <none> to fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "fdea56c2d1f5499087b939ec9f9dc070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: HEALTHY } } }
I20260204 08:52:46.243798 1457 leader_election.cc:290] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:46.244091 762 catalog_manager.cc:5654] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 reported cstate change: term changed from 0 to 1, leader changed from <none> to fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "fdea56c2d1f5499087b939ec9f9dc070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } }
I20260204 08:52:46.244714 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7fdfc8c910a7462684ea737615bd2870" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:46.244954 1022 raft_consensus.cc:2468] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:46.244933 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7fdfc8c910a7462684ea737615bd2870" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:46.245142 1159 raft_consensus.cc:2468] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:46.245257 762 catalog_manager.cc:5746] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737) does not have the latest schema for tablet cff3c1305f694fba837c0ced9cba71b0 (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
I20260204 08:52:46.245635 1232 leader_election.cc:304] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:46.245546 762 catalog_manager.cc:5654] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 reported cstate change: term changed from 0 to 1, leader changed from <none> to fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "fdea56c2d1f5499087b939ec9f9dc070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: HEALTHY } } }
I20260204 08:52:46.245985 762 catalog_manager.cc:5654] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 reported cstate change: term changed from 0 to 1, leader changed from <none> to fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "fdea56c2d1f5499087b939ec9f9dc070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: HEALTHY } } }
I20260204 08:52:46.246389 1457 raft_consensus.cc:2804] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:46.246498 1457 raft_consensus.cc:493] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:46.246618 1457 raft_consensus.cc:3060] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:46.249480 1457 raft_consensus.cc:515] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:46.251253 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7fdfc8c910a7462684ea737615bd2870" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:46.251478 1022 raft_consensus.cc:3060] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:46.251680 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7fdfc8c910a7462684ea737615bd2870" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:46.251894 1159 raft_consensus.cc:3060] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:46.254303 1022 raft_consensus.cc:2468] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:46.254837 1232 leader_election.cc:304] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:46.255228 1159 raft_consensus.cc:2468] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:46.255592 1468 raft_consensus.cc:2804] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:46.255764 1468 raft_consensus.cc:697] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Becoming Leader. State: Replica: 1234acd7d0db405085a5d95795007134, State: Running, Role: LEADER
I20260204 08:52:46.256071 1468 consensus_queue.cc:237] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:46.255821 1457 leader_election.cc:290] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:46.269909 762 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41804:
table { table_name: "default.Table_279632cf491c4198ad9b91a114c3fc33" } modify_external_catalogs: true
I20260204 08:52:46.270171 762 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41804:
table { table_name: "default.Table_279632cf491c4198ad9b91a114c3fc33" } modify_external_catalogs: true
I20260204 08:52:46.284781 1022 raft_consensus.cc:1275] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Refusing update from remote peer fdea56c2d1f5499087b939ec9f9dc070: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260204 08:52:46.284968 1296 raft_consensus.cc:1275] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Refusing update from remote peer fdea56c2d1f5499087b939ec9f9dc070: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260204 08:52:46.285936 1469 consensus_queue.cc:1048] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:46.286657 1469 consensus_queue.cc:1048] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:46.327366 1493 tablet.cc:1722] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:46.336835 1490 mvcc.cc:204] Tried to move back new op lower bound from 7250719401092190208 to 7250719398693244928. Current Snapshot: MvccSnapshot[applied={T|T < 7250719401092190208}]
I20260204 08:52:46.339751 1494 tablet.cc:1722] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:46.342034 1496 tablet.cc:1722] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:46.361699 755 catalog_manager.cc:3485] renamed table default.table_798df4bc6a0640c4b26f8fe2b3444d6b [id=893f125488f745a5b0bc9ec3d07da00b] in HMS: new name default.table_9f93a995778f4fe6a4667cff0ef494b9
W20260204 08:52:46.380410 1460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260204 08:52:46.380867 1460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260204 08:52:46.380976 1460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260204 08:52:46.395880 1460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260204 08:52:46.396014 1460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260204 08:52:46.396102 1460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260204 08:52:46.396160 1460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260204 08:52:46.413489 1460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:42471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.104.190
--webserver_port=40787
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0efbebd487e7d0c7f21ab0d5a91e0fa70046af83
build type FASTDEBUG
built by None at 04 Feb 2026 08:43:20 UTC on 24a791456cd2
build id 10340
ASAN enabled
I20260204 08:52:46.415843 1460 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260204 08:52:46.418270 1460 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260204 08:52:46.436050 1501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:46.436090 1500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260204 08:52:46.448096 754 catalog_manager.cc:3485] renamed table default.table_3956d1827061406b8d0a913345864f74 [id=105a9508f80c4190a40ab4ab66af2872] in HMS: new name default.table_f388b768c4b14290a90dd29962cc3fc4
W20260204 08:52:46.458441 1503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260204 08:52:46.460465 1460 server_base.cc:1060] running on GCE node
I20260204 08:52:46.461514 1460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260204 08:52:46.473413 1460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260204 08:52:46.474720 1460 hybrid_clock.cc:648] HybridClock initialized: now 1770195166474642 us; error 92 us; skew 500 ppm
I20260204 08:52:46.475288 1460 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260204 08:52:46.490535 1460 webserver.cc:492] Webserver started at http://127.0.104.190:40787/ using document root <none> and password file <none>
I20260204 08:52:46.491742 1460 fs_manager.cc:362] Metadata directory not provided
I20260204 08:52:46.491930 1460 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260204 08:52:46.504631 1460 fs_manager.cc:714] Time spent opening directory manager: real 0.010s user 0.006s sys 0.000s
I20260204 08:52:46.516044 1509 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260204 08:52:46.518415 1460 fs_manager.cc:730] Time spent opening block manager: real 0.011s user 0.005s sys 0.004s
I20260204 08:52:46.518688 1460 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data,/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34"
format_stamp: "Formatted at 2026-02-04 08:52:41 on dist-test-slave-vp4h"
I20260204 08:52:46.519912 1460 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260204 08:52:46.524180 761 catalog_manager.cc:3485] renamed table default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82] in HMS: new name default.table_e7fe420574dc40cdb70c4602f5a6e0db
I20260204 08:52:46.562436 759 catalog_manager.cc:2174] created HMS catalog entry for table [id=76307bf8c30c4654abb6343e24eed807]
I20260204 08:52:46.566743 1468 consensus_queue.cc:1048] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:46.569123 759 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41804 (request call id 21) took 1293 ms. Trace:
I20260204 08:52:46.569264 759 rpcz_store.cc:276] 0204 08:52:45.275203 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:45.389687 (+114484us) service_pool.cc:225] Handling call
0204 08:52:46.137484 (+747797us) catalog_manager.cc:2101] Acquired catalog manager lock
0204 08:52:46.137968 (+ 484us) catalog_manager.cc:2147] Created new table and tablet info
0204 08:52:46.562389 (+424421us) catalog_manager.cc:2173] Created new table in HMS catalog
0204 08:52:46.568792 (+ 6403us) catalog_manager.cc:2207] Wrote table and tablets to system table
0204 08:52:46.568905 (+ 113us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0204 08:52:46.569054 (+ 149us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:46.563251 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0204 08:52:46.563465 (+ 214us) write_op.cc:432] Acquiring schema lock in shared mode
0204 08:52:46.563471 (+ 6us) write_op.cc:435] Acquired schema lock
0204 08:52:46.563473 (+ 2us) tablet.cc:660] Decoding operations
0204 08:52:46.563556 (+ 83us) write_op.cc:620] Acquiring the partition lock for write op
0204 08:52:46.563569 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0204 08:52:46.563572 (+ 3us) tablet.cc:683] Acquiring locks for 2 operations
0204 08:52:46.563655 (+ 83us) tablet.cc:699] Row locks acquired
0204 08:52:46.563658 (+ 3us) write_op.cc:260] PREPARE: finished
0204 08:52:46.563701 (+ 43us) write_op.cc:270] Start()
0204 08:52:46.563733 (+ 32us) write_op.cc:276] Timestamp: P: 1770195166563652 usec, L: 0
0204 08:52:46.563735 (+ 2us) op_driver.cc:348] REPLICATION: starting
0204 08:52:46.563917 (+ 182us) log.cc:844] Serialized 432 byte log entry
0204 08:52:46.567305 (+ 3388us) op_driver.cc:464] REPLICATION: finished
0204 08:52:46.567534 (+ 229us) write_op.cc:301] APPLY: starting
0204 08:52:46.567551 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
0204 08:52:46.567599 (+ 48us) tablet.cc:1368] finished BulkCheckPresence
0204 08:52:46.567625 (+ 26us) tablet.cc:1370] starting ApplyRowOperation cycle
0204 08:52:46.567745 (+ 120us) tablet.cc:1381] finished ApplyRowOperation cycle
0204 08:52:46.567782 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0204 08:52:46.567789 (+ 7us) write_op.cc:312] APPLY: finished
0204 08:52:46.567888 (+ 99us) log.cc:844] Serialized 28 byte log entry
0204 08:52:46.567958 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:46.567981 (+ 23us) write_op.cc:454] Released schema lock
0204 08:52:46.567988 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0204 08:52:46.568074 (+ 86us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:46.568078 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":386042,"Hive Metastore.run_cpu_time_us":335,"Hive Metastore.run_wall_time_us":38131,"spinlock_wait_cycles":1035776,"child_traces":[["op",{"apply.queue_time_us":156,"apply.run_cpu_time_us":548,"apply.run_wall_time_us":548,"num_ops":2,"prepare.queue_time_us":65,"prepare.run_cpu_time_us":797,"prepare.run_wall_time_us":798,"raft.queue_time_us":570,"raft.run_cpu_time_us":349,"raft.run_wall_time_us":348,"replication_time_us":3537}]]}
I20260204 08:52:46.571758 759 catalog_manager.cc:5654] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1234acd7d0db405085a5d95795007134 (127.0.104.131). New cstate: current_term: 1 leader_uuid: "1234acd7d0db405085a5d95795007134" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: UNKNOWN } } }
I20260204 08:52:46.583458 1460 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260204 08:52:46.584781 1460 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260204 08:52:46.585136 1460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260204 08:52:46.588106 759 catalog_manager.cc:5746] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675) does not have the latest schema for tablet cff3c1305f694fba837c0ced9cba71b0 (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
I20260204 08:52:46.589800 1421 consensus_queue.cc:1048] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:46.594617 1002 tablet_service.cc:1505] Processing CreateTablet for tablet 9c96a74505f448e5ad1b8cc373ee82e4 (DEFAULT_TABLE table=default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.595110 1002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c96a74505f448e5ad1b8cc373ee82e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.599607 1139 tablet_service.cc:1505] Processing CreateTablet for tablet 9c96a74505f448e5ad1b8cc373ee82e4 (DEFAULT_TABLE table=default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.599992 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c96a74505f448e5ad1b8cc373ee82e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.600808 1275 tablet_service.cc:1505] Processing CreateTablet for tablet 9c96a74505f448e5ad1b8cc373ee82e4 (DEFAULT_TABLE table=default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.601233 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c96a74505f448e5ad1b8cc373ee82e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.610590 759 catalog_manager.cc:6199] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421] alter complete (version 1)
I20260204 08:52:46.612726 758 catalog_manager.cc:2174] created HMS catalog entry for table [id=180667e663144b7ea79638946d6c76cc]
I20260204 08:52:46.614532 759 master_service.cc:954] ReplaceTablet: received request to replace tablet 4452a7c31cdc4144bc61a36f401dcde2 from {username='slave'} at 127.0.0.1:41804
I20260204 08:52:46.616189 1517 tablet_bootstrap.cc:492] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:46.618778 1517 tablet_bootstrap.cc:654] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:46.622735 758 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41804 (request call id 22) took 1311 ms. Trace:
I20260204 08:52:46.622881 758 rpcz_store.cc:276] 0204 08:52:45.311004 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:45.457691 (+146687us) service_pool.cc:225] Handling call
0204 08:52:46.138497 (+680806us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x61200001e0c8. stack: 00007f462b2aca86 00007f462b2ac840 00007f462a2873ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f462b2953bd 00007f462b294fde 00007f463eef724d 00007f463eef71ec 00007f463f106f88 00007f463eec6463 00007f463ee22667 00007f463ee2dc7f 00007f463f031eea 00007f4635b307c6
0204 08:52:46.138706 (+ 209us) catalog_manager.cc:2101] Acquired catalog manager lock
0204 08:52:46.139092 (+ 386us) catalog_manager.cc:2147] Created new table and tablet info
0204 08:52:46.612686 (+473594us) catalog_manager.cc:2173] Created new table in HMS catalog
0204 08:52:46.622416 (+ 9730us) catalog_manager.cc:2207] Wrote table and tablets to system table
0204 08:52:46.622540 (+ 124us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0204 08:52:46.622658 (+ 118us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:46.617069 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0204 08:52:46.617193 (+ 124us) write_op.cc:432] Acquiring schema lock in shared mode
0204 08:52:46.617199 (+ 6us) write_op.cc:435] Acquired schema lock
0204 08:52:46.617202 (+ 3us) tablet.cc:660] Decoding operations
0204 08:52:46.617301 (+ 99us) write_op.cc:620] Acquiring the partition lock for write op
0204 08:52:46.617316 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0204 08:52:46.617326 (+ 10us) tablet.cc:683] Acquiring locks for 2 operations
0204 08:52:46.617411 (+ 85us) tablet.cc:699] Row locks acquired
0204 08:52:46.617414 (+ 3us) write_op.cc:260] PREPARE: finished
0204 08:52:46.617460 (+ 46us) write_op.cc:270] Start()
0204 08:52:46.617495 (+ 35us) write_op.cc:276] Timestamp: P: 1770195166617401 usec, L: 0
0204 08:52:46.617499 (+ 4us) op_driver.cc:348] REPLICATION: starting
0204 08:52:46.617722 (+ 223us) log.cc:844] Serialized 432 byte log entry
0204 08:52:46.621629 (+ 3907us) op_driver.cc:464] REPLICATION: finished
0204 08:52:46.621759 (+ 130us) write_op.cc:301] APPLY: starting
0204 08:52:46.621781 (+ 22us) tablet.cc:1365] starting BulkCheckPresence
0204 08:52:46.621838 (+ 57us) tablet.cc:1368] finished BulkCheckPresence
0204 08:52:46.621884 (+ 46us) tablet.cc:1370] starting ApplyRowOperation cycle
0204 08:52:46.622004 (+ 120us) tablet.cc:1381] finished ApplyRowOperation cycle
0204 08:52:46.622061 (+ 57us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0204 08:52:46.622069 (+ 8us) write_op.cc:312] APPLY: finished
0204 08:52:46.622168 (+ 99us) log.cc:844] Serialized 28 byte log entry
0204 08:52:46.622246 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:46.622272 (+ 26us) write_op.cc:454] Released schema lock
0204 08:52:46.622279 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"Hive Metastore.queue_time_us":423148,"Hive Metastore.run_cpu_time_us":235,"Hive Metastore.run_wall_time_us":50093,"spinlock_wait_cycles":2951552,"child_traces":[["op",{"apply.queue_time_us":46,"apply.run_cpu_time_us":597,"apply.run_wall_time_us":598,"num_ops":2,"prepare.queue_time_us":3713,"raft.queue_time_us":1860,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":262,"replication_time_us":4095}]]}
I20260204 08:52:46.627619 1517 tablet_bootstrap.cc:492] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:46.627877 1517 ts_tablet_manager.cc:1403] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.008s
I20260204 08:52:46.628832 1517 raft_consensus.cc:359] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.629390 1517 raft_consensus.cc:385] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:46.629554 1517 raft_consensus.cc:740] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:46.629899 1517 consensus_queue.cc:260] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.635085 1275 tablet_service.cc:1552] Processing DeleteTablet for tablet 4452a7c31cdc4144bc61a36f401dcde2 with delete_type TABLET_DATA_DELETED (replaced by tablet 043cdafd4b11463bae674695e7a74854) from {username='slave'} at 127.0.0.1:38330
I20260204 08:52:46.638468 1140 tablet_service.cc:1552] Processing DeleteTablet for tablet 4452a7c31cdc4144bc61a36f401dcde2 with delete_type TABLET_DATA_DELETED (replaced by tablet 043cdafd4b11463bae674695e7a74854) from {username='slave'} at 127.0.0.1:52546
I20260204 08:52:46.641328 1000 tablet_service.cc:1505] Processing CreateTablet for tablet 35b597aac0264d36a44ae37c6ffb945e (DEFAULT_TABLE table=default.table_7bcbc3cfbc41406cba3ed79125a46e82 [id=180667e663144b7ea79638946d6c76cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.641865 1000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35b597aac0264d36a44ae37c6ffb945e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.642391 1138 tablet_service.cc:1505] Processing CreateTablet for tablet 35b597aac0264d36a44ae37c6ffb945e (DEFAULT_TABLE table=default.table_7bcbc3cfbc41406cba3ed79125a46e82 [id=180667e663144b7ea79638946d6c76cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.642809 1138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35b597aac0264d36a44ae37c6ffb945e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.649706 1517 ts_tablet_manager.cc:1434] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.022s user 0.002s sys 0.000s
I20260204 08:52:46.650372 1276 tablet_service.cc:1505] Processing CreateTablet for tablet 35b597aac0264d36a44ae37c6ffb945e (DEFAULT_TABLE table=default.table_7bcbc3cfbc41406cba3ed79125a46e82 [id=180667e663144b7ea79638946d6c76cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.650839 1276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35b597aac0264d36a44ae37c6ffb945e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.665472 1000 tablet_service.cc:1552] Processing DeleteTablet for tablet 4452a7c31cdc4144bc61a36f401dcde2 with delete_type TABLET_DATA_DELETED (replaced by tablet 043cdafd4b11463bae674695e7a74854) from {username='slave'} at 127.0.0.1:39350
I20260204 08:52:46.670522 758 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41804:
table { table_name: "default.Table_73e876d60c444a5daa316e4bf7158614" } modify_external_catalogs: true
I20260204 08:52:46.671941 758 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41804:
table { table_name: "default.Table_73e876d60c444a5daa316e4bf7158614" } modify_external_catalogs: true
I20260204 08:52:46.671092 759 catalog_manager.cc:5941] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Sending DeleteTablet for 3 replicas of tablet 4452a7c31cdc4144bc61a36f401dcde2
I20260204 08:52:46.673466 1001 tablet_service.cc:1552] Processing DeleteTablet for tablet 4452a7c31cdc4144bc61a36f401dcde2 with delete_type TABLET_DATA_DELETED (replaced by tablet 043cdafd4b11463bae674695e7a74854) from {username='slave'} at 127.0.0.1:39350
I20260204 08:52:46.673808 1138 tablet_service.cc:1552] Processing DeleteTablet for tablet 4452a7c31cdc4144bc61a36f401dcde2 with delete_type TABLET_DATA_DELETED (replaced by tablet 043cdafd4b11463bae674695e7a74854) from {username='slave'} at 127.0.0.1:52546
I20260204 08:52:46.674305 1523 tablet_replica.cc:333] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:52:46.674533 1523 raft_consensus.cc:2243] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:46.674810 1523 raft_consensus.cc:2272] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:46.675966 1523 ts_tablet_manager.cc:1916] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:46.678313 759 catalog_manager.cc:6724] ReplaceTablet: tablet 4452a7c31cdc4144bc61a36f401dcde2 deleted and replaced by tablet 043cdafd4b11463bae674695e7a74854
I20260204 08:52:46.672585 1517 tablet_bootstrap.cc:492] T 35b597aac0264d36a44ae37c6ffb945e P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:46.678493 759 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:41804 (request call id 27) took 1185 ms. Trace:
I20260204 08:52:46.678575 759 rpcz_store.cc:276] 0204 08:52:45.493355 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:46.614453 (+1121098us) service_pool.cc:225] Handling call
0204 08:52:46.678434 (+ 63981us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:46.624230 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0204 08:52:46.624383 (+ 153us) write_op.cc:432] Acquiring schema lock in shared mode
0204 08:52:46.624388 (+ 5us) write_op.cc:435] Acquired schema lock
0204 08:52:46.624389 (+ 1us) tablet.cc:660] Decoding operations
0204 08:52:46.624506 (+ 117us) write_op.cc:620] Acquiring the partition lock for write op
0204 08:52:46.624519 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0204 08:52:46.624524 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0204 08:52:46.624656 (+ 132us) tablet.cc:699] Row locks acquired
0204 08:52:46.624659 (+ 3us) write_op.cc:260] PREPARE: finished
0204 08:52:46.624700 (+ 41us) write_op.cc:270] Start()
0204 08:52:46.624733 (+ 33us) write_op.cc:276] Timestamp: P: 1770195166624647 usec, L: 0
0204 08:52:46.624736 (+ 3us) op_driver.cc:348] REPLICATION: starting
0204 08:52:46.627920 (+ 3184us) log.cc:844] Serialized 661 byte log entry
0204 08:52:46.628231 (+ 311us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x616000041254. stack: 00007f462b2aca86 00007f462b2ac840 00007f462a2873ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f46371ac922 00007f463858f875 00007f463858e399 00007f46385967c0 00007f46385965cd 0000000000a8970c 00007f462b2f2daf 00007f462b2f60cb 00007f462b2f5f48 0000000000a8970c
0204 08:52:46.631840 (+ 3609us) op_driver.cc:464] REPLICATION: finished
0204 08:52:46.632372 (+ 532us) write_op.cc:301] APPLY: starting
0204 08:52:46.632391 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
0204 08:52:46.632438 (+ 47us) tablet.cc:1368] finished BulkCheckPresence
0204 08:52:46.632440 (+ 2us) tablet.cc:1370] starting ApplyRowOperation cycle
0204 08:52:46.632546 (+ 106us) tablet.cc:1381] finished ApplyRowOperation cycle
0204 08:52:46.632606 (+ 60us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0204 08:52:46.632613 (+ 7us) write_op.cc:312] APPLY: finished
0204 08:52:46.632694 (+ 81us) log.cc:844] Serialized 28 byte log entry
0204 08:52:46.632764 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:46.632783 (+ 19us) write_op.cc:454] Released schema lock
0204 08:52:46.632791 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0204 08:52:46.632864 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:46.632869 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":468,"apply.run_cpu_time_us":488,"apply.run_wall_time_us":488,"num_ops":2,"prepare.queue_time_us":8625,"prepare.run_cpu_time_us":1254,"prepare.run_wall_time_us":5300,"raft.queue_time_us":713,"raft.run_cpu_time_us":342,"raft.run_wall_time_us":342,"replication_time_us":7074,"spinlock_wait_cycles":6750592}]]}
I20260204 08:52:46.680881 1517 tablet_bootstrap.cc:654] T 35b597aac0264d36a44ae37c6ffb945e P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:46.657081 1521 tablet_replica.cc:333] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:52:46.681809 759 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41804:
name: "default.Table_27cce8dee4984465be213744abfadc9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:46.683367 1521 raft_consensus.cc:2243] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:52:46.684134 1521 raft_consensus.cc:2272] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:46.685168 1521 ts_tablet_manager.cc:1916] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:46.686518 1138 tablet_service.cc:1505] Processing CreateTablet for tablet 043cdafd4b11463bae674695e7a74854 (DEFAULT_TABLE table=default.table_798df4bc6a0640c4b26f8fe2b3444d6b [id=893f125488f745a5b0bc9ec3d07da00b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.686998 1138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 043cdafd4b11463bae674695e7a74854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.674882 1275 tablet_service.cc:1552] Processing DeleteTablet for tablet 4452a7c31cdc4144bc61a36f401dcde2 with delete_type TABLET_DATA_DELETED (replaced by tablet 043cdafd4b11463bae674695e7a74854) from {username='slave'} at 127.0.0.1:38330
I20260204 08:52:46.685312 1000 tablet_service.cc:1505] Processing CreateTablet for tablet 043cdafd4b11463bae674695e7a74854 (DEFAULT_TABLE table=default.table_798df4bc6a0640c4b26f8fe2b3444d6b [id=893f125488f745a5b0bc9ec3d07da00b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.687860 1000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 043cdafd4b11463bae674695e7a74854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.694654 1453 consensus_queue.cc:1048] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:46.687198 1519 tablet_bootstrap.cc:492] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:46.699636 1519 tablet_bootstrap.cc:654] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:46.704649 1275 tablet_service.cc:1505] Processing CreateTablet for tablet 043cdafd4b11463bae674695e7a74854 (DEFAULT_TABLE table=default.table_798df4bc6a0640c4b26f8fe2b3444d6b [id=893f125488f745a5b0bc9ec3d07da00b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.705147 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 043cdafd4b11463bae674695e7a74854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.707222 1517 tablet_bootstrap.cc:492] T 35b597aac0264d36a44ae37c6ffb945e P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
W20260204 08:52:46.693423 747 consensus_peers.cc:597] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 -> Peer ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471): Couldn't send request to peer ebeeb5d4823b41b3a8e37546ff2f6e34. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.190:42471: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260204 08:52:46.715402 1517 ts_tablet_manager.cc:1403] T 35b597aac0264d36a44ae37c6ffb945e P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.003s
I20260204 08:52:46.716183 1519 tablet_bootstrap.cc:492] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:46.716506 1519 ts_tablet_manager.cc:1403] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.029s user 0.004s sys 0.000s
I20260204 08:52:46.717648 1519 raft_consensus.cc:359] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.717941 1519 raft_consensus.cc:385] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:46.718096 1519 raft_consensus.cc:740] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:46.718456 1519 consensus_queue.cc:260] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.721956 1519 ts_tablet_manager.cc:1434] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260204 08:52:46.724182 1519 tablet_bootstrap.cc:492] T 35b597aac0264d36a44ae37c6ffb945e P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:46.722689 1523 ts_tablet_manager.cc:1929] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:46.725505 1523 log.cc:1199] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/4452a7c31cdc4144bc61a36f401dcde2
I20260204 08:52:46.726799 1523 ts_tablet_manager.cc:1950] T 4452a7c31cdc4144bc61a36f401dcde2 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:52:46.727331 757 catalog_manager.cc:2174] created HMS catalog entry for table [id=5f9f5b0850774ff9b8ecb93e141ff768]
I20260204 08:52:46.727667 1517 raft_consensus.cc:359] T 35b597aac0264d36a44ae37c6ffb945e P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.727959 1517 raft_consensus.cc:385] T 35b597aac0264d36a44ae37c6ffb945e P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:46.728065 1517 raft_consensus.cc:740] T 35b597aac0264d36a44ae37c6ffb945e P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:46.728533 1517 consensus_queue.cc:260] T 35b597aac0264d36a44ae37c6ffb945e P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.729478 1517 ts_tablet_manager.cc:1434] T 35b597aac0264d36a44ae37c6ffb945e P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20260204 08:52:46.735733 757 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41804 (request call id 20) took 1490 ms. Trace:
I20260204 08:52:46.735872 757 rpcz_store.cc:276] 0204 08:52:45.245430 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:45.359544 (+114114us) service_pool.cc:225] Handling call
0204 08:52:46.137368 (+777824us) catalog_manager.cc:2101] Acquired catalog manager lock
0204 08:52:46.137926 (+ 558us) catalog_manager.cc:2147] Created new table and tablet info
0204 08:52:46.727297 (+589371us) catalog_manager.cc:2173] Created new table in HMS catalog
0204 08:52:46.735381 (+ 8084us) catalog_manager.cc:2207] Wrote table and tablets to system table
0204 08:52:46.735527 (+ 146us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0204 08:52:46.735658 (+ 131us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:46.728363 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0204 08:52:46.728509 (+ 146us) write_op.cc:432] Acquiring schema lock in shared mode
0204 08:52:46.728517 (+ 8us) write_op.cc:435] Acquired schema lock
0204 08:52:46.728520 (+ 3us) tablet.cc:660] Decoding operations
0204 08:52:46.728652 (+ 132us) write_op.cc:620] Acquiring the partition lock for write op
0204 08:52:46.728667 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0204 08:52:46.728674 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0204 08:52:46.728766 (+ 92us) tablet.cc:699] Row locks acquired
0204 08:52:46.728769 (+ 3us) write_op.cc:260] PREPARE: finished
0204 08:52:46.728812 (+ 43us) write_op.cc:270] Start()
0204 08:52:46.728851 (+ 39us) write_op.cc:276] Timestamp: P: 1770195166728788 usec, L: 0
0204 08:52:46.728855 (+ 4us) op_driver.cc:348] REPLICATION: starting
0204 08:52:46.729080 (+ 225us) log.cc:844] Serialized 432 byte log entry
0204 08:52:46.733917 (+ 4837us) op_driver.cc:464] REPLICATION: finished
0204 08:52:46.734055 (+ 138us) write_op.cc:301] APPLY: starting
0204 08:52:46.734074 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
0204 08:52:46.734132 (+ 58us) tablet.cc:1368] finished BulkCheckPresence
0204 08:52:46.734159 (+ 27us) tablet.cc:1370] starting ApplyRowOperation cycle
0204 08:52:46.734262 (+ 103us) tablet.cc:1381] finished ApplyRowOperation cycle
0204 08:52:46.734298 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0204 08:52:46.734305 (+ 7us) write_op.cc:312] APPLY: finished
0204 08:52:46.734402 (+ 97us) log.cc:844] Serialized 28 byte log entry
0204 08:52:46.734483 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:46.734508 (+ 25us) write_op.cc:454] Released schema lock
0204 08:52:46.734515 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0204 08:52:46.734593 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:46.734596 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":474379,"Hive Metastore.run_cpu_time_us":290,"Hive Metastore.run_wall_time_us":114526,"mutex_wait_us":1269,"spinlock_wait_cycles":16640,"child_traces":[["op",{"apply.queue_time_us":88,"apply.run_cpu_time_us":526,"apply.run_wall_time_us":526,"num_ops":2,"prepare.queue_time_us":210,"prepare.run_cpu_time_us":858,"prepare.run_wall_time_us":860,"raft.queue_time_us":1333,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":290,"replication_time_us":5028}]]}
I20260204 08:52:46.737394 1521 ts_tablet_manager.cc:1929] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:46.737535 1521 log.cc:1199] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/4452a7c31cdc4144bc61a36f401dcde2
I20260204 08:52:46.737967 1521 ts_tablet_manager.cc:1950] T 4452a7c31cdc4144bc61a36f401dcde2 P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:52:46.739266 749 catalog_manager.cc:4985] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): tablet 4452a7c31cdc4144bc61a36f401dcde2 (table default.table_798df4bc6a0640c4b26f8fe2b3444d6b [id=893f125488f745a5b0bc9ec3d07da00b]) successfully deleted
W20260204 08:52:46.740067 749 catalog_manager.cc:4952] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): delete failed for tablet 4452a7c31cdc4144bc61a36f401dcde2 because the tablet was not found. No further retry: Not found: Tablet not found: 4452a7c31cdc4144bc61a36f401dcde2
I20260204 08:52:46.741346 1519 tablet_bootstrap.cc:654] T 35b597aac0264d36a44ae37c6ffb945e P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:46.741767 749 catalog_manager.cc:4985] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): tablet 4452a7c31cdc4144bc61a36f401dcde2 (table default.table_798df4bc6a0640c4b26f8fe2b3444d6b [id=893f125488f745a5b0bc9ec3d07da00b]) successfully deleted
I20260204 08:52:46.745307 1517 tablet_bootstrap.cc:492] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:46.747136 1517 tablet_bootstrap.cc:654] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
W20260204 08:52:46.748397 749 catalog_manager.cc:4952] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): delete failed for tablet 4452a7c31cdc4144bc61a36f401dcde2 because the tablet was not found. No further retry: Not found: Tablet not found: 4452a7c31cdc4144bc61a36f401dcde2
I20260204 08:52:46.749761 1453 consensus_queue.cc:1048] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260204 08:52:46.755431 1002 tablet_service.cc:1505] Processing CreateTablet for tablet bf91146e638f43e0b493a5732f6174be (DEFAULT_TABLE table=default.table_c97013dd16f84d8ba7cd528295f5ad49 [id=5f9f5b0850774ff9b8ecb93e141ff768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.755983 1002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf91146e638f43e0b493a5732f6174be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.756326 1469 consensus_queue.cc:1048] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260204 08:52:46.759941 1276 tablet_service.cc:1505] Processing CreateTablet for tablet bf91146e638f43e0b493a5732f6174be (DEFAULT_TABLE table=default.table_c97013dd16f84d8ba7cd528295f5ad49 [id=5f9f5b0850774ff9b8ecb93e141ff768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.760485 1276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf91146e638f43e0b493a5732f6174be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.763959 757 master_service.cc:954] ReplaceTablet: received request to replace tablet cff3c1305f694fba837c0ced9cba71b0 from {username='slave'} at 127.0.0.1:41804
I20260204 08:52:46.768779 1457 consensus_queue.cc:1048] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:46.769815 1139 tablet_service.cc:1505] Processing CreateTablet for tablet bf91146e638f43e0b493a5732f6174be (DEFAULT_TABLE table=default.table_c97013dd16f84d8ba7cd528295f5ad49 [id=5f9f5b0850774ff9b8ecb93e141ff768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.770341 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf91146e638f43e0b493a5732f6174be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.776724 1518 tablet_bootstrap.cc:492] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:46.778190 1519 tablet_bootstrap.cc:492] T 35b597aac0264d36a44ae37c6ffb945e P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:46.778934 1519 ts_tablet_manager.cc:1403] T 35b597aac0264d36a44ae37c6ffb945e P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.055s user 0.003s sys 0.004s
I20260204 08:52:46.780093 1519 raft_consensus.cc:359] T 35b597aac0264d36a44ae37c6ffb945e P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.781453 1522 tablet_replica.cc:333] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:52:46.783170 1522 raft_consensus.cc:2243] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:46.784518 1522 raft_consensus.cc:2272] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:46.782213 1518 tablet_bootstrap.cc:654] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:46.787849 1517 tablet_bootstrap.cc:492] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:46.787935 1522 ts_tablet_manager.cc:1916] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:46.788045 1517 ts_tablet_manager.cc:1403] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.043s user 0.001s sys 0.007s
I20260204 08:52:46.788861 1517 raft_consensus.cc:359] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.789074 1517 raft_consensus.cc:385] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:46.789219 1517 raft_consensus.cc:740] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:46.789513 1517 consensus_queue.cc:260] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.789692 757 catalog_manager.cc:5941] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Sending DeleteTablet for 3 replicas of tablet cff3c1305f694fba837c0ced9cba71b0
I20260204 08:52:46.791301 1139 tablet_service.cc:1552] Processing DeleteTablet for tablet cff3c1305f694fba837c0ced9cba71b0 with delete_type TABLET_DATA_DELETED (replaced by tablet 96ed25c622d946dfb4531ae603a80bad) from {username='slave'} at 127.0.0.1:52546
I20260204 08:52:46.791486 1275 tablet_service.cc:1552] Processing DeleteTablet for tablet cff3c1305f694fba837c0ced9cba71b0 with delete_type TABLET_DATA_DELETED (replaced by tablet 96ed25c622d946dfb4531ae603a80bad) from {username='slave'} at 127.0.0.1:38330
I20260204 08:52:46.791977 1517 ts_tablet_manager.cc:1434] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260204 08:52:46.792223 1521 tablet_replica.cc:333] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:52:46.792498 1521 raft_consensus.cc:2243] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:46.792809 1521 raft_consensus.cc:2272] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:46.793949 1517 tablet_bootstrap.cc:492] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:46.794108 1521 ts_tablet_manager.cc:1916] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:46.795928 1517 tablet_bootstrap.cc:654] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:46.792207 1000 tablet_service.cc:1552] Processing DeleteTablet for tablet cff3c1305f694fba837c0ced9cba71b0 with delete_type TABLET_DATA_DELETED (replaced by tablet 96ed25c622d946dfb4531ae603a80bad) from {username='slave'} at 127.0.0.1:39350
I20260204 08:52:46.799803 1523 tablet_replica.cc:333] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:52:46.800067 1523 raft_consensus.cc:2243] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:46.800415 1523 raft_consensus.cc:2272] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:46.801370 1523 ts_tablet_manager.cc:1916] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:46.801822 1000 tablet_service.cc:1505] Processing CreateTablet for tablet 96ed25c622d946dfb4531ae603a80bad (DEFAULT_TABLE table=default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.802260 1000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96ed25c622d946dfb4531ae603a80bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.802363 1518 tablet_bootstrap.cc:492] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:46.802513 1518 ts_tablet_manager.cc:1403] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.046s user 0.003s sys 0.003s
I20260204 08:52:46.803354 1518 raft_consensus.cc:359] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.803572 1518 raft_consensus.cc:385] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:46.803664 1518 raft_consensus.cc:740] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:46.806077 1519 raft_consensus.cc:385] T 35b597aac0264d36a44ae37c6ffb945e P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:46.806252 1519 raft_consensus.cc:740] T 35b597aac0264d36a44ae37c6ffb945e P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:46.807758 1519 consensus_queue.cc:260] T 35b597aac0264d36a44ae37c6ffb945e P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.809103 1521 ts_tablet_manager.cc:1929] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260204 08:52:46.809612 1522 ts_tablet_manager.cc:1929] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:46.809751 1522 log.cc:1199] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/4452a7c31cdc4144bc61a36f401dcde2
I20260204 08:52:46.809686 1518 consensus_queue.cc:260] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.810223 1522 ts_tablet_manager.cc:1950] T 4452a7c31cdc4144bc61a36f401dcde2 P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:52:46.810428 1139 tablet_service.cc:1505] Processing CreateTablet for tablet 96ed25c622d946dfb4531ae603a80bad (DEFAULT_TABLE table=default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.810863 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96ed25c622d946dfb4531ae603a80bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.811369 1519 ts_tablet_manager.cc:1434] T 35b597aac0264d36a44ae37c6ffb945e P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.032s user 0.000s sys 0.002s
I20260204 08:52:46.811733 1519 tablet_bootstrap.cc:492] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:46.812464 757 catalog_manager.cc:6724] ReplaceTablet: tablet cff3c1305f694fba837c0ced9cba71b0 deleted and replaced by tablet 96ed25c622d946dfb4531ae603a80bad
I20260204 08:52:46.812796 757 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:41804 (request call id 30) took 1293 ms. Trace:
I20260204 08:52:46.813148 757 rpcz_store.cc:276] 0204 08:52:45.519421 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:46.763852 (+1244431us) service_pool.cc:225] Handling call
0204 08:52:46.787355 (+ 23503us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:46.787365 (+ 10us) write_op.cc:454] Released schema lock
0204 08:52:46.812755 (+ 25390us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:46.765566 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0204 08:52:46.765690 (+ 124us) write_op.cc:432] Acquiring schema lock in shared mode
0204 08:52:46.765696 (+ 6us) write_op.cc:435] Acquired schema lock
0204 08:52:46.765698 (+ 2us) tablet.cc:660] Decoding operations
0204 08:52:46.765818 (+ 120us) write_op.cc:620] Acquiring the partition lock for write op
0204 08:52:46.765832 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0204 08:52:46.765837 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0204 08:52:46.765944 (+ 107us) tablet.cc:699] Row locks acquired
0204 08:52:46.765946 (+ 2us) write_op.cc:260] PREPARE: finished
0204 08:52:46.765987 (+ 41us) write_op.cc:270] Start()
0204 08:52:46.766027 (+ 40us) write_op.cc:276] Timestamp: P: 1770195166765960 usec, L: 0
0204 08:52:46.766030 (+ 3us) op_driver.cc:348] REPLICATION: starting
0204 08:52:46.766268 (+ 238us) log.cc:844] Serialized 661 byte log entry
0204 08:52:46.771359 (+ 5091us) op_driver.cc:464] REPLICATION: finished
0204 08:52:46.771506 (+ 147us) write_op.cc:301] APPLY: starting
0204 08:52:46.771530 (+ 24us) tablet.cc:1365] starting BulkCheckPresence
0204 08:52:46.771587 (+ 57us) tablet.cc:1368] finished BulkCheckPresence
0204 08:52:46.771615 (+ 28us) tablet.cc:1370] starting ApplyRowOperation cycle
0204 08:52:46.771735 (+ 120us) tablet.cc:1381] finished ApplyRowOperation cycle
0204 08:52:46.771808 (+ 73us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0204 08:52:46.771817 (+ 9us) write_op.cc:312] APPLY: finished
0204 08:52:46.771905 (+ 88us) log.cc:844] Serialized 28 byte log entry
0204 08:52:46.771977 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:46.772002 (+ 25us) write_op.cc:454] Released schema lock
0204 08:52:46.772009 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":93,"apply.run_cpu_time_us":560,"apply.run_wall_time_us":558,"num_ops":2,"prepare.queue_time_us":54,"prepare.run_cpu_time_us":829,"prepare.run_wall_time_us":830,"raft.queue_time_us":1265,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":290,"replication_time_us":5282}]]}
I20260204 08:52:46.814586 1519 tablet_bootstrap.cc:654] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:46.820673 1523 ts_tablet_manager.cc:1929] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260204 08:52:46.820879 1523 log.cc:1199] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/cff3c1305f694fba837c0ced9cba71b0
I20260204 08:52:46.821400 1523 ts_tablet_manager.cc:1950] T cff3c1305f694fba837c0ced9cba71b0 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:52:46.822615 749 catalog_manager.cc:4985] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): tablet cff3c1305f694fba837c0ced9cba71b0 (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]) successfully deleted
I20260204 08:52:46.827840 1519 tablet_bootstrap.cc:492] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:46.828020 1519 ts_tablet_manager.cc:1403] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.004s
I20260204 08:52:46.828703 1518 ts_tablet_manager.cc:1434] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.026s user 0.001s sys 0.002s
I20260204 08:52:46.829407 1519 raft_consensus.cc:359] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.829612 1519 raft_consensus.cc:385] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:46.829706 1519 raft_consensus.cc:740] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:46.830219 1519 consensus_queue.cc:260] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.830814 1423 consensus_queue.cc:1048] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:46.831019 1519 ts_tablet_manager.cc:1434] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260204 08:52:46.831319 1519 tablet_bootstrap.cc:492] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:46.834236 1519 tablet_bootstrap.cc:654] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:46.835873 1518 tablet_bootstrap.cc:492] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:46.838469 1518 tablet_bootstrap.cc:654] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:46.841904 749 catalog_manager.cc:4985] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737): tablet 4452a7c31cdc4144bc61a36f401dcde2 (table default.table_798df4bc6a0640c4b26f8fe2b3444d6b [id=893f125488f745a5b0bc9ec3d07da00b]) successfully deleted
W20260204 08:52:46.843752 749 catalog_manager.cc:4952] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737): delete failed for tablet 4452a7c31cdc4144bc61a36f401dcde2 because the tablet was not found. No further retry: Not found: Tablet not found: 4452a7c31cdc4144bc61a36f401dcde2
I20260204 08:52:46.844166 1519 tablet_bootstrap.cc:492] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:46.844354 1519 ts_tablet_manager.cc:1403] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.009s
I20260204 08:52:46.844548 1522 tablet_replica.cc:333] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:52:46.844753 1522 raft_consensus.cc:2243] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:52:46.845132 1519 raft_consensus.cc:359] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.845322 1519 raft_consensus.cc:385] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:46.845341 1522 raft_consensus.cc:2272] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:46.845395 1519 raft_consensus.cc:740] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:46.845676 1519 consensus_queue.cc:260] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.811266 1275 tablet_service.cc:1505] Processing CreateTablet for tablet 96ed25c622d946dfb4531ae603a80bad (DEFAULT_TABLE table=default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.847319 1522 ts_tablet_manager.cc:1916] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:46.847481 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96ed25c622d946dfb4531ae603a80bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.849284 1519 ts_tablet_manager.cc:1434] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260204 08:52:46.823807 1421 raft_consensus.cc:493] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:46.851215 1421 raft_consensus.cc:515] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.851898 1421 leader_election.cc:290] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:46.838884 1457 consensus_queue.cc:1048] T 80f72b647ee643afabcc5cafa9c43b13 P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:46.852475 1519 tablet_bootstrap.cc:492] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:46.854894 1519 tablet_bootstrap.cc:654] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:46.855190 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c96a74505f448e5ad1b8cc373ee82e4" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:46.855429 1159 raft_consensus.cc:2468] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:46.855708 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c96a74505f448e5ad1b8cc373ee82e4" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:46.856215 1022 raft_consensus.cc:2468] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:46.857260 1232 leader_election.cc:304] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:46.857693 1421 raft_consensus.cc:2804] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:46.857810 1421 raft_consensus.cc:493] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:46.857884 1421 raft_consensus.cc:3060] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:46.842581 1517 tablet_bootstrap.cc:492] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:46.860102 1517 ts_tablet_manager.cc:1403] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.066s user 0.022s sys 0.003s
I20260204 08:52:46.861095 1421 raft_consensus.cc:515] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.861384 1521 log.cc:1199] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/cff3c1305f694fba837c0ced9cba71b0
I20260204 08:52:46.861971 1521 ts_tablet_manager.cc:1950] T cff3c1305f694fba837c0ced9cba71b0 P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:52:46.862573 1020 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c96a74505f448e5ad1b8cc373ee82e4" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:46.862794 1020 raft_consensus.cc:3060] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:46.863114 749 catalog_manager.cc:4985] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): tablet cff3c1305f694fba837c0ced9cba71b0 (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]) successfully deleted
I20260204 08:52:46.863127 1421 leader_election.cc:290] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:46.865583 1020 raft_consensus.cc:2468] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:46.867304 1232 leader_election.cc:304] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:46.868000 1421 raft_consensus.cc:2804] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:46.868396 1421 raft_consensus.cc:697] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Becoming Leader. State: Replica: 1234acd7d0db405085a5d95795007134, State: Running, Role: LEADER
I20260204 08:52:46.861006 1517 raft_consensus.cc:359] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.868665 1517 raft_consensus.cc:385] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:46.868777 1517 raft_consensus.cc:740] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:46.868947 1421 consensus_queue.cc:237] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.869814 1517 consensus_queue.cc:260] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.870260 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c96a74505f448e5ad1b8cc373ee82e4" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:46.870591 1159 raft_consensus.cc:3060] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:46.871869 1517 ts_tablet_manager.cc:1434] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.012s user 0.000s sys 0.002s
I20260204 08:52:46.872236 1517 tablet_bootstrap.cc:492] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:46.873957 1159 raft_consensus.cc:2468] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:46.888789 757 master_service.cc:954] ReplaceTablet: received request to replace tablet 1a3763eb94ac42858c2986e3c11965f0 from {username='slave'} at 127.0.0.1:41804
I20260204 08:52:46.893811 1518 tablet_bootstrap.cc:492] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:46.894016 1518 ts_tablet_manager.cc:1403] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.058s user 0.004s sys 0.000s
I20260204 08:52:46.894806 1518 raft_consensus.cc:359] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.895054 1518 raft_consensus.cc:385] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:46.895159 1518 raft_consensus.cc:740] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:46.895447 1518 consensus_queue.cc:260] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.897135 1423 consensus_queue.cc:1048] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:46.901136 1518 ts_tablet_manager.cc:1434] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20260204 08:52:46.904062 757 catalog_manager.cc:5941] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Sending DeleteTablet for 3 replicas of tablet 1a3763eb94ac42858c2986e3c11965f0
I20260204 08:52:46.904099 1518 tablet_bootstrap.cc:492] T 043cdafd4b11463bae674695e7a74854 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:46.905411 1000 tablet_service.cc:1552] Processing DeleteTablet for tablet 1a3763eb94ac42858c2986e3c11965f0 with delete_type TABLET_DATA_DELETED (replaced by tablet ebed8bb53e7947e08d0c960b13bacd8d) from {username='slave'} at 127.0.0.1:39350
I20260204 08:52:46.905738 1523 tablet_replica.cc:333] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:52:46.905834 757 catalog_manager.cc:6724] ReplaceTablet: tablet 1a3763eb94ac42858c2986e3c11965f0 deleted and replaced by tablet ebed8bb53e7947e08d0c960b13bacd8d
I20260204 08:52:46.905877 1139 tablet_service.cc:1552] Processing DeleteTablet for tablet 1a3763eb94ac42858c2986e3c11965f0 with delete_type TABLET_DATA_DELETED (replaced by tablet ebed8bb53e7947e08d0c960b13bacd8d) from {username='slave'} at 127.0.0.1:52546
I20260204 08:52:46.905942 1523 raft_consensus.cc:2243] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:46.906145 1523 raft_consensus.cc:2272] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:46.906594 757 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:41804 (request call id 31) took 1383 ms. Trace:
I20260204 08:52:46.906692 757 rpcz_store.cc:276] 0204 08:52:45.522544 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:46.888719 (+1366175us) service_pool.cc:225] Handling call
0204 08:52:46.905937 (+ 17218us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:46.893817 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0204 08:52:46.893962 (+ 145us) write_op.cc:432] Acquiring schema lock in shared mode
0204 08:52:46.893969 (+ 7us) write_op.cc:435] Acquired schema lock
0204 08:52:46.893972 (+ 3us) tablet.cc:660] Decoding operations
0204 08:52:46.894097 (+ 125us) write_op.cc:620] Acquiring the partition lock for write op
0204 08:52:46.894112 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0204 08:52:46.894117 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0204 08:52:46.894204 (+ 87us) tablet.cc:699] Row locks acquired
0204 08:52:46.894206 (+ 2us) write_op.cc:260] PREPARE: finished
0204 08:52:46.894250 (+ 44us) write_op.cc:270] Start()
0204 08:52:46.894295 (+ 45us) write_op.cc:276] Timestamp: P: 1770195166894209 usec, L: 0
0204 08:52:46.894299 (+ 4us) op_driver.cc:348] REPLICATION: starting
0204 08:52:46.894499 (+ 200us) log.cc:844] Serialized 661 byte log entry
0204 08:52:46.899611 (+ 5112us) op_driver.cc:464] REPLICATION: finished
0204 08:52:46.899797 (+ 186us) write_op.cc:301] APPLY: starting
0204 08:52:46.899815 (+ 18us) tablet.cc:1365] starting BulkCheckPresence
0204 08:52:46.899860 (+ 45us) tablet.cc:1368] finished BulkCheckPresence
0204 08:52:46.899880 (+ 20us) tablet.cc:1370] starting ApplyRowOperation cycle
0204 08:52:46.900006 (+ 126us) tablet.cc:1381] finished ApplyRowOperation cycle
0204 08:52:46.900090 (+ 84us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0204 08:52:46.900098 (+ 8us) write_op.cc:312] APPLY: finished
0204 08:52:46.900180 (+ 82us) log.cc:844] Serialized 28 byte log entry
0204 08:52:46.900248 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:46.900271 (+ 23us) write_op.cc:454] Released schema lock
0204 08:52:46.900277 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0204 08:52:46.900379 (+ 102us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:46.900383 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":125,"apply.run_cpu_time_us":573,"apply.run_wall_time_us":572,"num_ops":2,"prepare.queue_time_us":1143,"prepare.run_cpu_time_us":804,"prepare.run_wall_time_us":804,"raft.queue_time_us":1292,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":265,"replication_time_us":5263}]]}
I20260204 08:52:46.911916 1523 ts_tablet_manager.cc:1916] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:46.907459 1518 tablet_bootstrap.cc:654] T 043cdafd4b11463bae674695e7a74854 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:46.917986 1453 consensus_queue.cc:1048] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:46.904892 1275 tablet_service.cc:1552] Processing DeleteTablet for tablet 1a3763eb94ac42858c2986e3c11965f0 with delete_type TABLET_DATA_DELETED (replaced by tablet ebed8bb53e7947e08d0c960b13bacd8d) from {username='slave'} at 127.0.0.1:38330
I20260204 08:52:46.923302 1521 tablet_replica.cc:333] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:52:46.923522 1521 raft_consensus.cc:2243] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:46.923764 1521 raft_consensus.cc:2272] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:46.919518 1457 consensus_queue.cc:1048] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:46.925488 1521 ts_tablet_manager.cc:1916] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:46.927023 1517 tablet_bootstrap.cc:654] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:46.932022 1521 ts_tablet_manager.cc:1929] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:46.932170 1521 log.cc:1199] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/1a3763eb94ac42858c2986e3c11965f0
I20260204 08:52:46.933276 1000 tablet_service.cc:1505] Processing CreateTablet for tablet ebed8bb53e7947e08d0c960b13bacd8d (DEFAULT_TABLE table=default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.933367 757 catalog_manager.cc:5654] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1234acd7d0db405085a5d95795007134 (127.0.104.131). New cstate: current_term: 1 leader_uuid: "1234acd7d0db405085a5d95795007134" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: HEALTHY } } }
I20260204 08:52:46.933732 1000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebed8bb53e7947e08d0c960b13bacd8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.934273 1519 tablet_bootstrap.cc:492] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:46.934427 1519 ts_tablet_manager.cc:1403] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.082s user 0.025s sys 0.026s
I20260204 08:52:46.935330 1519 raft_consensus.cc:359] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.935544 1519 raft_consensus.cc:385] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:46.935626 1519 raft_consensus.cc:740] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:46.935868 1519 consensus_queue.cc:260] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.936625 1519 ts_tablet_manager.cc:1434] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260204 08:52:46.937024 1139 tablet_service.cc:1505] Processing CreateTablet for tablet ebed8bb53e7947e08d0c960b13bacd8d (DEFAULT_TABLE table=default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.937451 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebed8bb53e7947e08d0c960b13bacd8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.938344 1522 ts_tablet_manager.cc:1929] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260204 08:52:46.938477 1522 log.cc:1199] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/cff3c1305f694fba837c0ced9cba71b0
I20260204 08:52:46.938930 1522 ts_tablet_manager.cc:1950] T cff3c1305f694fba837c0ced9cba71b0 P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:52:46.939903 1522 tablet_replica.cc:333] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:52:46.940106 1522 raft_consensus.cc:2243] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:52:46.940452 1522 raft_consensus.cc:2272] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:46.940591 1275 tablet_service.cc:1505] Processing CreateTablet for tablet ebed8bb53e7947e08d0c960b13bacd8d (DEFAULT_TABLE table=default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:46.940999 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebed8bb53e7947e08d0c960b13bacd8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:46.941898 749 catalog_manager.cc:4985] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737): tablet cff3c1305f694fba837c0ced9cba71b0 (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]) successfully deleted
I20260204 08:52:46.947211 1522 ts_tablet_manager.cc:1916] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:46.952134 757 catalog_manager.cc:5746] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675) does not have the latest schema for tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
I20260204 08:52:46.954305 1519 tablet_bootstrap.cc:492] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:46.955598 1523 ts_tablet_manager.cc:1929] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:46.957324 1523 log.cc:1199] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/1a3763eb94ac42858c2986e3c11965f0
I20260204 08:52:46.957821 1523 ts_tablet_manager.cc:1950] T 1a3763eb94ac42858c2986e3c11965f0 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:52:46.958250 1521 ts_tablet_manager.cc:1950] T 1a3763eb94ac42858c2986e3c11965f0 P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:52:46.956069 1519 tablet_bootstrap.cc:654] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:46.961297 1517 tablet_bootstrap.cc:492] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:46.961462 1517 ts_tablet_manager.cc:1403] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.089s user 0.000s sys 0.007s
I20260204 08:52:46.959225 749 catalog_manager.cc:4985] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): tablet 1a3763eb94ac42858c2986e3c11965f0 (table default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294]) successfully deleted
I20260204 08:52:46.961748 749 catalog_manager.cc:4985] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): tablet 1a3763eb94ac42858c2986e3c11965f0 (table default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294]) successfully deleted
I20260204 08:52:46.962805 1517 raft_consensus.cc:359] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.963008 1517 raft_consensus.cc:385] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:46.963083 1517 raft_consensus.cc:740] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:46.963284 1517 consensus_queue.cc:260] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
W20260204 08:52:46.964174 757 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 96ed25c622d946dfb4531ae603a80bad consensus state has no leader
I20260204 08:52:46.968853 757 catalog_manager.cc:5746] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491) does not have the latest schema for tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
I20260204 08:52:46.969118 1517 ts_tablet_manager.cc:1434] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20260204 08:52:46.969486 1517 tablet_bootstrap.cc:492] T ebed8bb53e7947e08d0c960b13bacd8d P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:46.970642 1538 consensus_queue.cc:1048] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:46.974409 1518 tablet_bootstrap.cc:492] T 043cdafd4b11463bae674695e7a74854 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:46.974581 1518 ts_tablet_manager.cc:1403] T 043cdafd4b11463bae674695e7a74854 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.071s user 0.008s sys 0.002s
I20260204 08:52:46.975826 1453 raft_consensus.cc:493] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:46.975916 1518 raft_consensus.cc:359] T 043cdafd4b11463bae674695e7a74854 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.975997 1453 raft_consensus.cc:515] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.976150 1518 raft_consensus.cc:385] T 043cdafd4b11463bae674695e7a74854 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:46.976253 1518 raft_consensus.cc:740] T 043cdafd4b11463bae674695e7a74854 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:46.976522 1518 consensus_queue.cc:260] T 043cdafd4b11463bae674695e7a74854 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.976866 1453 leader_election.cc:290] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:46.977357 1517 tablet_bootstrap.cc:654] T ebed8bb53e7947e08d0c960b13bacd8d P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:46.977736 1518 ts_tablet_manager.cc:1434] T 043cdafd4b11463bae674695e7a74854 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260204 08:52:46.978060 1518 tablet_bootstrap.cc:492] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:46.978044 1295 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35b597aac0264d36a44ae37c6ffb945e" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:46.978327 1295 raft_consensus.cc:2468] T 35b597aac0264d36a44ae37c6ffb945e P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:46.979141 1519 tablet_bootstrap.cc:492] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:46.979311 1519 ts_tablet_manager.cc:1403] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.002s
I20260204 08:52:46.979642 1096 leader_election.cc:304] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:46.979859 1518 tablet_bootstrap.cc:654] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:46.980115 1519 raft_consensus.cc:359] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.980360 1519 raft_consensus.cc:385] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:46.980437 1519 raft_consensus.cc:740] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:46.980671 1519 consensus_queue.cc:260] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.981225 1519 ts_tablet_manager.cc:1434] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260204 08:52:46.981529 1453 raft_consensus.cc:2804] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:46.981652 1453 raft_consensus.cc:493] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:46.981734 1453 raft_consensus.cc:3060] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:46.983721 1518 tablet_bootstrap.cc:492] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:46.983887 1518 ts_tablet_manager.cc:1403] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260204 08:52:46.984781 1518 raft_consensus.cc:359] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.985014 1518 raft_consensus.cc:385] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:46.985103 1518 raft_consensus.cc:740] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:46.985441 1453 raft_consensus.cc:515] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.986603 757 catalog_manager.cc:5746] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491) does not have the latest schema for tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
I20260204 08:52:46.987390 1295 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35b597aac0264d36a44ae37c6ffb945e" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:46.987577 1295 raft_consensus.cc:3060] T 35b597aac0264d36a44ae37c6ffb945e P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:46.989821 1518 consensus_queue.cc:260] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.990284 1020 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35b597aac0264d36a44ae37c6ffb945e" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:46.990522 1020 raft_consensus.cc:3060] T 35b597aac0264d36a44ae37c6ffb945e P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:46.989568 1522 ts_tablet_manager.cc:1929] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:46.991083 1522 log.cc:1199] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/1a3763eb94ac42858c2986e3c11965f0
I20260204 08:52:46.991559 1522 ts_tablet_manager.cc:1950] T 1a3763eb94ac42858c2986e3c11965f0 P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
W20260204 08:52:46.992584 749 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 96ed25c622d946dfb4531ae603a80bad consensus state has no leader
I20260204 08:52:46.992605 1518 ts_tablet_manager.cc:1434] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260204 08:52:46.993970 1518 tablet_bootstrap.cc:492] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:46.993942 1020 raft_consensus.cc:2468] T 35b597aac0264d36a44ae37c6ffb945e P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:46.994853 1096 leader_election.cc:304] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:46.995272 1469 raft_consensus.cc:2804] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:46.995404 1469 raft_consensus.cc:697] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Becoming Leader. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Running, Role: LEADER
I20260204 08:52:46.995685 1469 consensus_queue.cc:237] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:46.995821 1518 tablet_bootstrap.cc:654] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:46.998071 749 catalog_manager.cc:4985] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737): tablet 1a3763eb94ac42858c2986e3c11965f0 (table default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294]) successfully deleted
I20260204 08:52:47.000204 1295 raft_consensus.cc:2468] T 35b597aac0264d36a44ae37c6ffb945e P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:47.001341 757 master_service.cc:954] ReplaceTablet: received request to replace tablet 7fdfc8c910a7462684ea737615bd2870 from {username='slave'} at 127.0.0.1:41804
I20260204 08:52:47.003538 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35b597aac0264d36a44ae37c6ffb945e" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:47.003914 1022 raft_consensus.cc:2376] T 35b597aac0264d36a44ae37c6ffb945e P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1. Re-sending same reply.
I20260204 08:52:47.006531 1453 leader_election.cc:290] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:47.012781 757 catalog_manager.cc:5941] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Sending DeleteTablet for 3 replicas of tablet 7fdfc8c910a7462684ea737615bd2870
I20260204 08:52:47.012789 1453 consensus_queue.cc:1048] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:47.013634 1275 tablet_service.cc:1552] Processing DeleteTablet for tablet 7fdfc8c910a7462684ea737615bd2870 with delete_type TABLET_DATA_DELETED (replaced by tablet 60e7158619d5432594db8192ca00efbb) from {username='slave'} at 127.0.0.1:38330
I20260204 08:52:47.013947 1521 tablet_replica.cc:333] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:52:47.014174 1521 raft_consensus.cc:2243] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:52:47.014433 1521 pending_rounds.cc:70] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134: Trying to abort 1 pending ops.
I20260204 08:52:47.014616 1521 pending_rounds.cc:77] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7250719400987570176 op_type: NO_OP noop_request { }
I20260204 08:52:47.014755 1521 raft_consensus.cc:2889] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260204 08:52:47.014899 1521 raft_consensus.cc:2272] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:47.015830 1521 ts_tablet_manager.cc:1916] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:47.022609 1521 ts_tablet_manager.cc:1929] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:47.022770 1521 log.cc:1199] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/7fdfc8c910a7462684ea737615bd2870
I20260204 08:52:47.023200 1521 ts_tablet_manager.cc:1950] T 7fdfc8c910a7462684ea737615bd2870 P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:52:47.024431 749 catalog_manager.cc:4985] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): tablet 7fdfc8c910a7462684ea737615bd2870 (table default.table_43ccd0dac1b24e20880a930959aa280a [id=31385bebafdd42fe89d79858c81a5a18]) successfully deleted
I20260204 08:52:47.025972 1000 tablet_service.cc:1552] Processing DeleteTablet for tablet 7fdfc8c910a7462684ea737615bd2870 with delete_type TABLET_DATA_DELETED (replaced by tablet 60e7158619d5432594db8192ca00efbb) from {username='slave'} at 127.0.0.1:39350
I20260204 08:52:47.026289 1523 tablet_replica.cc:333] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:52:47.026512 1523 raft_consensus.cc:2243] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:47.026707 1523 raft_consensus.cc:2272] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:47.027333 1523 ts_tablet_manager.cc:1916] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:47.028980 1453 raft_consensus.cc:493] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:47.030910 1457 consensus_queue.cc:1048] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:47.032287 1453 raft_consensus.cc:515] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.033717 1523 ts_tablet_manager.cc:1929] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260204 08:52:47.034929 1518 tablet_bootstrap.cc:492] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:47.035131 1518 ts_tablet_manager.cc:1403] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.042s user 0.004s sys 0.000s
I20260204 08:52:47.035411 1523 log.cc:1199] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/7fdfc8c910a7462684ea737615bd2870
I20260204 08:52:47.035993 1518 raft_consensus.cc:359] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.035801 1523 ts_tablet_manager.cc:1950] T 7fdfc8c910a7462684ea737615bd2870 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:52:47.036209 1518 raft_consensus.cc:385] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:47.036319 1518 raft_consensus.cc:740] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:47.036592 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf91146e638f43e0b493a5732f6174be" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:47.036849 1022 raft_consensus.cc:2468] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:47.037307 1139 tablet_service.cc:1552] Processing DeleteTablet for tablet 7fdfc8c910a7462684ea737615bd2870 with delete_type TABLET_DATA_DELETED (replaced by tablet 60e7158619d5432594db8192ca00efbb) from {username='slave'} at 127.0.0.1:52546
I20260204 08:52:47.037303 749 catalog_manager.cc:4985] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): tablet 7fdfc8c910a7462684ea737615bd2870 (table default.table_43ccd0dac1b24e20880a930959aa280a [id=31385bebafdd42fe89d79858c81a5a18]) successfully deleted
I20260204 08:52:47.037592 1522 tablet_replica.cc:333] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:52:47.037808 1522 raft_consensus.cc:2243] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:47.037994 1522 raft_consensus.cc:2272] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:47.038116 1096 leader_election.cc:304] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:47.038625 1522 ts_tablet_manager.cc:1916] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:47.042543 1423 consensus_queue.cc:1048] T 81f37ac4f50a440d8e7292bfa05a0a7d P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:47.043195 1423 raft_consensus.cc:2804] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:47.043361 1423 raft_consensus.cc:493] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:47.043516 1423 raft_consensus.cc:3060] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.046809 1453 leader_election.cc:290] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:47.054127 757 catalog_manager.cc:6724] ReplaceTablet: tablet 7fdfc8c910a7462684ea737615bd2870 deleted and replaced by tablet 60e7158619d5432594db8192ca00efbb
I20260204 08:52:47.054337 757 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:41804 (request call id 32) took 1529 ms. Trace:
I20260204 08:52:47.054286 1000 tablet_service.cc:1505] Processing CreateTablet for tablet 60e7158619d5432594db8192ca00efbb (DEFAULT_TABLE table=default.table_43ccd0dac1b24e20880a930959aa280a [id=31385bebafdd42fe89d79858c81a5a18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:47.054716 1000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60e7158619d5432594db8192ca00efbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:47.054426 757 rpcz_store.cc:276] 0204 08:52:45.524400 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.001228 (+1476828us) service_pool.cc:225] Handling call
0204 08:52:47.012475 (+ 11247us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:47.012482 (+ 7us) write_op.cc:454] Released schema lock
0204 08:52:47.054265 (+ 41783us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:47.004796 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0204 08:52:47.004935 (+ 139us) write_op.cc:432] Acquiring schema lock in shared mode
0204 08:52:47.004943 (+ 8us) write_op.cc:435] Acquired schema lock
0204 08:52:47.004945 (+ 2us) tablet.cc:660] Decoding operations
0204 08:52:47.005087 (+ 142us) write_op.cc:620] Acquiring the partition lock for write op
0204 08:52:47.005102 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0204 08:52:47.005107 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0204 08:52:47.005190 (+ 83us) tablet.cc:699] Row locks acquired
0204 08:52:47.005192 (+ 2us) write_op.cc:260] PREPARE: finished
0204 08:52:47.005234 (+ 42us) write_op.cc:270] Start()
0204 08:52:47.005280 (+ 46us) write_op.cc:276] Timestamp: P: 1770195167005184 usec, L: 0
0204 08:52:47.005284 (+ 4us) op_driver.cc:348] REPLICATION: starting
0204 08:52:47.005515 (+ 231us) log.cc:844] Serialized 661 byte log entry
0204 08:52:47.009390 (+ 3875us) op_driver.cc:464] REPLICATION: finished
0204 08:52:47.009520 (+ 130us) write_op.cc:301] APPLY: starting
0204 08:52:47.009537 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
0204 08:52:47.009584 (+ 47us) tablet.cc:1368] finished BulkCheckPresence
0204 08:52:47.009619 (+ 35us) tablet.cc:1370] starting ApplyRowOperation cycle
0204 08:52:47.009724 (+ 105us) tablet.cc:1381] finished ApplyRowOperation cycle
0204 08:52:47.009793 (+ 69us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0204 08:52:47.009800 (+ 7us) write_op.cc:312] APPLY: finished
0204 08:52:47.009884 (+ 84us) log.cc:844] Serialized 28 byte log entry
0204 08:52:47.009970 (+ 86us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:47.009993 (+ 23us) write_op.cc:454] Released schema lock
0204 08:52:47.010000 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":80,"apply.run_cpu_time_us":536,"apply.run_wall_time_us":536,"num_ops":2,"prepare.queue_time_us":2371,"prepare.run_cpu_time_us":850,"prepare.run_wall_time_us":851,"raft.queue_time_us":1138,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":259,"replication_time_us":4075}]]}
I20260204 08:52:47.056628 757 master_service.cc:954] ReplaceTablet: received request to replace tablet adcec26d4c104929b2880775a9cb29c5 from {username='slave'} at 127.0.0.1:41804
I20260204 08:52:47.057749 1275 tablet_service.cc:1505] Processing CreateTablet for tablet 60e7158619d5432594db8192ca00efbb (DEFAULT_TABLE table=default.table_43ccd0dac1b24e20880a930959aa280a [id=31385bebafdd42fe89d79858c81a5a18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:47.058209 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60e7158619d5432594db8192ca00efbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:47.064962 1518 consensus_queue.cc:260] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.066748 1518 ts_tablet_manager.cc:1434] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.031s user 0.003s sys 0.000s
I20260204 08:52:47.067658 1518 tablet_bootstrap.cc:492] T ebed8bb53e7947e08d0c960b13bacd8d P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:47.069512 1518 tablet_bootstrap.cc:654] T ebed8bb53e7947e08d0c960b13bacd8d P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:47.075304 1522 ts_tablet_manager.cc:1929] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260204 08:52:47.075490 1522 log.cc:1199] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/7fdfc8c910a7462684ea737615bd2870
I20260204 08:52:47.075868 1522 ts_tablet_manager.cc:1950] T 7fdfc8c910a7462684ea737615bd2870 P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:52:47.083182 1519 tablet_bootstrap.cc:492] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:47.085044 1519 tablet_bootstrap.cc:654] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:47.085129 757 catalog_manager.cc:5941] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Sending DeleteTablet for 3 replicas of tablet adcec26d4c104929b2880775a9cb29c5
I20260204 08:52:47.086411 1276 tablet_service.cc:1552] Processing DeleteTablet for tablet adcec26d4c104929b2880775a9cb29c5 with delete_type TABLET_DATA_DELETED (replaced by tablet fe0b2df895c643e7b69fb30a2b0b8e5c) from {username='slave'} at 127.0.0.1:38330
I20260204 08:52:47.086812 1457 raft_consensus.cc:493] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:47.086988 1457 raft_consensus.cc:515] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.087658 1457 leader_election.cc:290] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:47.088665 1002 tablet_service.cc:1552] Processing DeleteTablet for tablet adcec26d4c104929b2880775a9cb29c5 with delete_type TABLET_DATA_DELETED (replaced by tablet fe0b2df895c643e7b69fb30a2b0b8e5c) from {username='slave'} at 127.0.0.1:39350
I20260204 08:52:47.088965 1523 tablet_replica.cc:333] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:52:47.086825 1521 tablet_replica.cc:333] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:52:47.089182 1523 raft_consensus.cc:2243] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:47.089228 1521 raft_consensus.cc:2243] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:47.089388 1523 raft_consensus.cc:2272] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:47.089450 1521 raft_consensus.cc:2272] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:47.090535 1523 ts_tablet_manager.cc:1916] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:47.093869 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "043cdafd4b11463bae674695e7a74854" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:47.095295 1022 raft_consensus.cc:2468] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:47.096963 1521 ts_tablet_manager.cc:1916] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:47.097383 757 catalog_manager.cc:6724] ReplaceTablet: tablet adcec26d4c104929b2880775a9cb29c5 deleted and replaced by tablet fe0b2df895c643e7b69fb30a2b0b8e5c
I20260204 08:52:47.097505 757 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:41804 (request call id 33) took 1546 ms. Trace:
I20260204 08:52:47.097597 757 rpcz_store.cc:276] 0204 08:52:45.550588 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.056537 (+1505949us) service_pool.cc:225] Handling call
0204 08:52:47.084835 (+ 28298us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:47.084842 (+ 7us) write_op.cc:454] Released schema lock
0204 08:52:47.097475 (+ 12633us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:47.076629 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0204 08:52:47.076763 (+ 134us) write_op.cc:432] Acquiring schema lock in shared mode
0204 08:52:47.076770 (+ 7us) write_op.cc:435] Acquired schema lock
0204 08:52:47.076773 (+ 3us) tablet.cc:660] Decoding operations
0204 08:52:47.076896 (+ 123us) write_op.cc:620] Acquiring the partition lock for write op
0204 08:52:47.076911 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0204 08:52:47.076916 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0204 08:52:47.076991 (+ 75us) tablet.cc:699] Row locks acquired
0204 08:52:47.076993 (+ 2us) write_op.cc:260] PREPARE: finished
0204 08:52:47.077039 (+ 46us) write_op.cc:270] Start()
0204 08:52:47.077076 (+ 37us) write_op.cc:276] Timestamp: P: 1770195167076979 usec, L: 0
0204 08:52:47.077080 (+ 4us) op_driver.cc:348] REPLICATION: starting
0204 08:52:47.077323 (+ 243us) log.cc:844] Serialized 661 byte log entry
0204 08:52:47.081780 (+ 4457us) op_driver.cc:464] REPLICATION: finished
0204 08:52:47.081910 (+ 130us) write_op.cc:301] APPLY: starting
0204 08:52:47.081928 (+ 18us) tablet.cc:1365] starting BulkCheckPresence
0204 08:52:47.081979 (+ 51us) tablet.cc:1368] finished BulkCheckPresence
0204 08:52:47.082007 (+ 28us) tablet.cc:1370] starting ApplyRowOperation cycle
0204 08:52:47.082115 (+ 108us) tablet.cc:1381] finished ApplyRowOperation cycle
0204 08:52:47.082171 (+ 56us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0204 08:52:47.082177 (+ 6us) write_op.cc:312] APPLY: finished
0204 08:52:47.082261 (+ 84us) log.cc:844] Serialized 28 byte log entry
0204 08:52:47.082343 (+ 82us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:47.082367 (+ 24us) write_op.cc:454] Released schema lock
0204 08:52:47.082375 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":82,"apply.run_cpu_time_us":529,"apply.run_wall_time_us":528,"num_ops":2,"prepare.queue_time_us":18879,"prepare.run_cpu_time_us":927,"prepare.run_wall_time_us":927,"raft.queue_time_us":1241,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":262,"replication_time_us":4665}]]}
I20260204 08:52:47.098330 1275 tablet_service.cc:1505] Processing CreateTablet for tablet fe0b2df895c643e7b69fb30a2b0b8e5c (DEFAULT_TABLE table=default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:47.098886 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe0b2df895c643e7b69fb30a2b0b8e5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:47.099570 757 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41804:
table { table_name: "default.Table_9e950ede78084ef18f4e1051d9a42a90" } modify_external_catalogs: true
I20260204 08:52:47.099815 757 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41804:
table { table_name: "default.Table_9e950ede78084ef18f4e1051d9a42a90" } modify_external_catalogs: true
I20260204 08:52:47.100457 1232 leader_election.cc:304] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:47.100839 1457 raft_consensus.cc:2804] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:47.100927 1457 raft_consensus.cc:493] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:47.101033 1457 raft_consensus.cc:3060] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.102802 1523 ts_tablet_manager.cc:1929] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:47.102957 1523 log.cc:1199] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/adcec26d4c104929b2880775a9cb29c5
I20260204 08:52:47.103406 1523 ts_tablet_manager.cc:1950] T adcec26d4c104929b2880775a9cb29c5 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:52:47.104560 749 catalog_manager.cc:4985] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): tablet adcec26d4c104929b2880775a9cb29c5 (table default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82]) successfully deleted
I20260204 08:52:47.099414 1000 tablet_service.cc:1505] Processing CreateTablet for tablet fe0b2df895c643e7b69fb30a2b0b8e5c (DEFAULT_TABLE table=default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:47.105518 1000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe0b2df895c643e7b69fb30a2b0b8e5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:47.106151 1565 raft_consensus.cc:493] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:47.107577 1565 raft_consensus.cc:515] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.113255 1160 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "043cdafd4b11463bae674695e7a74854" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:47.113513 1160 raft_consensus.cc:2468] T 043cdafd4b11463bae674695e7a74854 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:47.119148 1423 raft_consensus.cc:515] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.119926 1423 leader_election.cc:290] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:47.121361 1538 consensus_queue.cc:1048] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260204 08:52:47.123796 1457 raft_consensus.cc:515] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.125761 1160 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "043cdafd4b11463bae674695e7a74854" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:47.125962 1160 raft_consensus.cc:3060] T 043cdafd4b11463bae674695e7a74854 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.125981 1457 leader_election.cc:290] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:47.126322 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "043cdafd4b11463bae674695e7a74854" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:47.126499 1022 raft_consensus.cc:3060] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.128870 1160 raft_consensus.cc:2468] T 043cdafd4b11463bae674695e7a74854 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:47.129391 1232 leader_election.cc:304] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:47.143728 1538 raft_consensus.cc:2804] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:47.143949 1538 raft_consensus.cc:697] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Becoming Leader. State: Replica: 1234acd7d0db405085a5d95795007134, State: Running, Role: LEADER
I20260204 08:52:47.145763 1538 consensus_queue.cc:237] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
W20260204 08:52:47.171499 1096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.104.129:36675, user_credentials={real_user=slave}} blocked reactor thread for 133358us
I20260204 08:52:47.183640 1022 raft_consensus.cc:2468] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:47.194716 1521 ts_tablet_manager.cc:1929] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:47.195029 1521 log.cc:1199] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/adcec26d4c104929b2880775a9cb29c5
I20260204 08:52:47.195724 1295 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "043cdafd4b11463bae674695e7a74854" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:47.198572 1295 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf91146e638f43e0b493a5732f6174be" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:47.198858 1295 raft_consensus.cc:2468] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:47.199488 749 catalog_manager.cc:4985] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737): tablet 7fdfc8c910a7462684ea737615bd2870 (table default.table_43ccd0dac1b24e20880a930959aa280a [id=31385bebafdd42fe89d79858c81a5a18]) successfully deleted
I20260204 08:52:47.205639 1517 tablet_bootstrap.cc:492] T ebed8bb53e7947e08d0c960b13bacd8d P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:47.205843 1517 ts_tablet_manager.cc:1403] T ebed8bb53e7947e08d0c960b13bacd8d P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.236s user 0.023s sys 0.084s
I20260204 08:52:47.207211 1517 raft_consensus.cc:359] T ebed8bb53e7947e08d0c960b13bacd8d P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.207454 1517 raft_consensus.cc:385] T ebed8bb53e7947e08d0c960b13bacd8d P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:47.207533 1517 raft_consensus.cc:740] T ebed8bb53e7947e08d0c960b13bacd8d P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:47.208020 1517 consensus_queue.cc:260] T ebed8bb53e7947e08d0c960b13bacd8d P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.208698 1517 ts_tablet_manager.cc:1434] T ebed8bb53e7947e08d0c960b13bacd8d P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260204 08:52:47.208994 1517 tablet_bootstrap.cc:492] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:47.210750 1517 tablet_bootstrap.cc:654] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:47.197752 1519 tablet_bootstrap.cc:492] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:47.216794 1519 ts_tablet_manager.cc:1403] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.134s user 0.005s sys 0.007s
I20260204 08:52:47.217823 1519 raft_consensus.cc:359] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.218159 1519 raft_consensus.cc:385] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:47.218305 1519 raft_consensus.cc:740] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:47.218636 1519 consensus_queue.cc:260] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.222462 1519 ts_tablet_manager.cc:1434] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260204 08:52:47.225857 1519 tablet_bootstrap.cc:492] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:47.197873 1565 leader_election.cc:290] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:47.227700 1521 ts_tablet_manager.cc:1950] T adcec26d4c104929b2880775a9cb29c5 P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:52:47.229346 749 catalog_manager.cc:4985] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): tablet adcec26d4c104929b2880775a9cb29c5 (table default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82]) successfully deleted
I20260204 08:52:47.235944 1518 tablet_bootstrap.cc:492] T ebed8bb53e7947e08d0c960b13bacd8d P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:47.236316 1518 ts_tablet_manager.cc:1403] T ebed8bb53e7947e08d0c960b13bacd8d P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.169s user 0.006s sys 0.033s
I20260204 08:52:47.237172 798 master_runner.cc:186] Error getting master registration for 127.0.104.190:42471: Network error: Client connection negotiation failed: client connection to 127.0.104.190:42471: connect: Connection refused (error 111),
I20260204 08:52:47.237367 798 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:52:47.237331 1518 raft_consensus.cc:359] T ebed8bb53e7947e08d0c960b13bacd8d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.237568 1518 raft_consensus.cc:385] T ebed8bb53e7947e08d0c960b13bacd8d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:47.237650 1518 raft_consensus.cc:740] T ebed8bb53e7947e08d0c960b13bacd8d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:47.238955 1140 tablet_service.cc:1552] Processing DeleteTablet for tablet adcec26d4c104929b2880775a9cb29c5 with delete_type TABLET_DATA_DELETED (replaced by tablet fe0b2df895c643e7b69fb30a2b0b8e5c) from {username='slave'} at 127.0.0.1:52546
I20260204 08:52:47.238917 1295 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf91146e638f43e0b493a5732f6174be" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:47.239147 1295 raft_consensus.cc:3060] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.239125 1139 tablet_service.cc:1505] Processing CreateTablet for tablet 60e7158619d5432594db8192ca00efbb (DEFAULT_TABLE table=default.table_43ccd0dac1b24e20880a930959aa280a [id=31385bebafdd42fe89d79858c81a5a18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:47.239621 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60e7158619d5432594db8192ca00efbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:47.240367 1522 tablet_replica.cc:333] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:52:47.240417 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf91146e638f43e0b493a5732f6174be" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:47.240577 1522 raft_consensus.cc:2243] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:52:47.240624 1022 raft_consensus.cc:3060] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.240953 1522 raft_consensus.cc:2272] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:47.239554 1140 tablet_service.cc:1505] Processing CreateTablet for tablet fe0b2df895c643e7b69fb30a2b0b8e5c (DEFAULT_TABLE table=default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:47.241950 1295 raft_consensus.cc:2468] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:47.242223 1522 ts_tablet_manager.cc:1916] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:47.242470 1096 leader_election.cc:304] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:47.242785 1423 raft_consensus.cc:2804] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:47.242707 1518 consensus_queue.cc:260] T ebed8bb53e7947e08d0c960b13bacd8d P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.242933 1423 raft_consensus.cc:697] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Becoming Leader. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Running, Role: LEADER
I20260204 08:52:47.243201 1423 consensus_queue.cc:237] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.243845 1140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe0b2df895c643e7b69fb30a2b0b8e5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:47.244911 1022 raft_consensus.cc:2468] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:47.245374 1518 ts_tablet_manager.cc:1434] T ebed8bb53e7947e08d0c960b13bacd8d P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260204 08:52:47.246003 1519 tablet_bootstrap.cc:654] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:47.260453 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "043cdafd4b11463bae674695e7a74854" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:47.260784 1159 raft_consensus.cc:2393] T 043cdafd4b11463bae674695e7a74854 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6d471268b9e643e5bbbf8bde1644a5ae in current term 1: Already voted for candidate 1234acd7d0db405085a5d95795007134 in this term.
I20260204 08:52:47.261344 958 leader_election.cc:304] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae; no voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070
I20260204 08:52:47.262063 1565 raft_consensus.cc:2749] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:52:47.263238 1517 tablet_bootstrap.cc:492] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:47.263458 1517 ts_tablet_manager.cc:1403] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.055s user 0.013s sys 0.022s
I20260204 08:52:47.263621 1519 tablet_bootstrap.cc:492] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:47.263790 1519 ts_tablet_manager.cc:1403] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.002s
I20260204 08:52:47.264341 1517 raft_consensus.cc:359] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.264535 1517 raft_consensus.cc:385] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:47.264534 1519 raft_consensus.cc:359] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.264631 1517 raft_consensus.cc:740] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:47.264721 1519 raft_consensus.cc:385] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:47.264844 1519 raft_consensus.cc:740] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:47.264858 1517 consensus_queue.cc:260] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.265805 1517 ts_tablet_manager.cc:1434] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260204 08:52:47.266208 1517 tablet_bootstrap.cc:492] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:47.266768 1519 consensus_queue.cc:260] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.267125 1518 tablet_bootstrap.cc:492] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:47.268868 1518 tablet_bootstrap.cc:654] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:47.269042 1517 tablet_bootstrap.cc:654] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:47.269066 1522 ts_tablet_manager.cc:1929] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:47.269316 1522 log.cc:1199] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/adcec26d4c104929b2880775a9cb29c5
I20260204 08:52:47.269797 1522 ts_tablet_manager.cc:1950] T adcec26d4c104929b2880775a9cb29c5 P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:52:47.271765 1518 tablet_bootstrap.cc:492] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:47.271972 1518 ts_tablet_manager.cc:1403] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260204 08:52:47.273481 749 catalog_manager.cc:4985] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737): tablet adcec26d4c104929b2880775a9cb29c5 (table default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82]) successfully deleted
I20260204 08:52:47.273545 1518 raft_consensus.cc:359] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.273720 1518 raft_consensus.cc:385] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:47.273801 1518 raft_consensus.cc:740] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:47.274013 1518 consensus_queue.cc:260] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.274554 1518 ts_tablet_manager.cc:1434] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260204 08:52:47.274592 1519 ts_tablet_manager.cc:1434] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260204 08:52:47.278554 1518 tablet_bootstrap.cc:492] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:47.280654 1518 tablet_bootstrap.cc:654] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
W20260204 08:52:47.284844 747 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:42471: connect: Connection refused (error 111) [suppressed 10 similar messages]
I20260204 08:52:47.285446 1518 tablet_bootstrap.cc:492] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:47.285633 1518 ts_tablet_manager.cc:1403] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20260204 08:52:47.286458 1518 raft_consensus.cc:359] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.286679 1518 raft_consensus.cc:385] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:47.286748 1518 raft_consensus.cc:740] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:47.287027 1518 consensus_queue.cc:260] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.290939 1518 ts_tablet_manager.cc:1434] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260204 08:52:47.302338 1565 raft_consensus.cc:493] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:47.302573 1565 raft_consensus.cc:515] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.303354 1565 leader_election.cc:290] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:47.303725 1468 raft_consensus.cc:493] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:47.303918 1468 raft_consensus.cc:515] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.304596 1468 leader_election.cc:290] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:47.305016 1517 tablet_bootstrap.cc:492] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:47.305229 1517 ts_tablet_manager.cc:1403] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.039s user 0.015s sys 0.018s
I20260204 08:52:47.305625 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ebed8bb53e7947e08d0c960b13bacd8d" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:47.305833 1159 raft_consensus.cc:2468] T ebed8bb53e7947e08d0c960b13bacd8d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 0.
I20260204 08:52:47.306074 1517 raft_consensus.cc:359] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.306180 1295 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ebed8bb53e7947e08d0c960b13bacd8d" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:47.306308 1517 raft_consensus.cc:385] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:47.306384 1295 raft_consensus.cc:2468] T ebed8bb53e7947e08d0c960b13bacd8d P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 0.
I20260204 08:52:47.306389 1517 raft_consensus.cc:740] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:47.306668 1517 consensus_queue.cc:260] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.306963 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "96ed25c622d946dfb4531ae603a80bad" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:47.307173 1022 raft_consensus.cc:2468] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:47.307184 1517 ts_tablet_manager.cc:1434] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260204 08:52:47.311100 958 leader_election.cc:304] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:47.311815 1232 leader_election.cc:304] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:47.312206 1457 raft_consensus.cc:2804] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:47.312348 1457 raft_consensus.cc:493] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:47.312438 1457 raft_consensus.cc:3060] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.315230 1457 raft_consensus.cc:515] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.316574 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "96ed25c622d946dfb4531ae603a80bad" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:47.316814 1159 raft_consensus.cc:2468] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:47.317559 1457 leader_election.cc:290] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:47.322031 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "96ed25c622d946dfb4531ae603a80bad" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:47.322255 1022 raft_consensus.cc:3060] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.324841 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "96ed25c622d946dfb4531ae603a80bad" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:47.325078 1159 raft_consensus.cc:3060] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.327809 1159 raft_consensus.cc:2468] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:47.328504 1232 leader_election.cc:304] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:47.328936 1468 raft_consensus.cc:2804] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:47.329099 1468 raft_consensus.cc:697] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Becoming Leader. State: Replica: 1234acd7d0db405085a5d95795007134, State: Running, Role: LEADER
I20260204 08:52:47.329362 1468 consensus_queue.cc:237] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.331218 1565 raft_consensus.cc:2804] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:47.331347 1565 raft_consensus.cc:493] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:47.331421 1565 raft_consensus.cc:3060] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.334681 1565 raft_consensus.cc:515] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.336119 1160 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ebed8bb53e7947e08d0c960b13bacd8d" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:47.336395 1160 raft_consensus.cc:3060] T ebed8bb53e7947e08d0c960b13bacd8d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.339143 1160 raft_consensus.cc:2468] T ebed8bb53e7947e08d0c960b13bacd8d P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 1.
I20260204 08:52:47.339948 958 leader_election.cc:304] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:47.340361 1420 raft_consensus.cc:2804] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:47.341768 1022 raft_consensus.cc:2468] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:47.344449 1565 leader_election.cc:290] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 election: Requested vote from peers fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:47.345526 1295 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ebed8bb53e7947e08d0c960b13bacd8d" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:47.345772 1295 raft_consensus.cc:3060] T ebed8bb53e7947e08d0c960b13bacd8d P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.349933 1295 raft_consensus.cc:2468] T ebed8bb53e7947e08d0c960b13bacd8d P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 1.
I20260204 08:52:47.352558 1420 raft_consensus.cc:697] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 LEADER]: Becoming Leader. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Running, Role: LEADER
I20260204 08:52:47.353107 1420 consensus_queue.cc:237] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.427194 728 master_runner.cc:186] Error getting master registration for 127.0.104.190:42471: Network error: Client connection negotiation failed: client connection to 127.0.104.190:42471: connect: Connection refused (error 111),
I20260204 08:52:47.427449 728 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:52:47.494864 1420 raft_consensus.cc:493] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:47.495141 1420 raft_consensus.cc:515] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.495908 1420 leader_election.cc:290] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:47.497007 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "60e7158619d5432594db8192ca00efbb" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:47.497287 1159 raft_consensus.cc:2468] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 0.
I20260204 08:52:47.498309 1295 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "60e7158619d5432594db8192ca00efbb" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:47.498567 1295 raft_consensus.cc:2468] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 0.
I20260204 08:52:47.499401 958 leader_election.cc:304] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:47.500130 1420 raft_consensus.cc:2804] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:47.500240 1420 raft_consensus.cc:493] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:47.500377 1420 raft_consensus.cc:3060] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.503607 1420 raft_consensus.cc:515] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.505142 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "60e7158619d5432594db8192ca00efbb" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:47.505409 1159 raft_consensus.cc:3060] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.508670 1159 raft_consensus.cc:2468] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 1.
I20260204 08:52:47.509373 958 leader_election.cc:304] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:47.509712 1606 raft_consensus.cc:2804] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:47.509866 1606 raft_consensus.cc:697] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 LEADER]: Becoming Leader. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Running, Role: LEADER
I20260204 08:52:47.510210 1606 consensus_queue.cc:237] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.510821 1295 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "60e7158619d5432594db8192ca00efbb" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:47.511036 1295 raft_consensus.cc:3060] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.514250 1295 raft_consensus.cc:2468] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 1.
I20260204 08:52:47.515270 1420 leader_election.cc:290] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 election: Requested vote from peers fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:47.551681 1628 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.190:42471 every 8 connection(s)
I20260204 08:52:47.551681 1460 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.190:42471
I20260204 08:52:47.555133 1460 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
I20260204 08:52:47.568058 1629 sys_catalog.cc:263] Verifying existing consensus state
I20260204 08:52:47.568944 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu as pid 1460
I20260204 08:52:47.575691 1629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Bootstrap starting.
I20260204 08:52:47.585131 1629 log.cc:826] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Log is configured to *not* fsync() on all Append() calls
I20260204 08:52:47.596400 1633 raft_consensus.cc:493] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:52:47.596735 1633 raft_consensus.cc:515] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:47.597918 1633 leader_election.cc:290] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005)
I20260204 08:52:47.598268 1629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260204 08:52:47.598977 1629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Bootstrap complete.
I20260204 08:52:47.599152 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1e719e457e94a5da71929720411ae69" candidate_term: 2 candidate_status { last_received { term: 1 index: 51 } } ignore_live_leader: false dest_uuid: "d083ecc9895345e5b06cd00ef0246c71" is_pre_election: true
I20260204 08:52:47.607481 1629 raft_consensus.cc:359] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:47.608527 1629 raft_consensus.cc:740] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebeeb5d4823b41b3a8e37546ff2f6e34, State: Initialized, Role: FOLLOWER
I20260204 08:52:47.609326 1629 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:47.611055 1634 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:47.611657 1634 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:47.612439 1629 sys_catalog.cc:565] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: configured and running, proceeding with master startup.
I20260204 08:52:47.612514 1592 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1e719e457e94a5da71929720411ae69" candidate_term: 2 candidate_status { last_received { term: 1 index: 51 } } ignore_live_leader: false dest_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" is_pre_election: true
I20260204 08:52:47.613057 1592 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1e719e457e94a5da71929720411ae69 in term 1.
I20260204 08:52:47.613936 817 leader_election.cc:304] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c1e719e457e94a5da71929720411ae69, ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: d083ecc9895345e5b06cd00ef0246c71
I20260204 08:52:47.614450 1633 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260204 08:52:47.614563 1633 raft_consensus.cc:493] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:52:47.614645 1633 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 1 FOLLOWER]: Advancing to term 2
I20260204 08:52:47.618026 1633 raft_consensus.cc:515] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:47.619583 1592 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1e719e457e94a5da71929720411ae69" candidate_term: 2 candidate_status { last_received { term: 1 index: 51 } } ignore_live_leader: false dest_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34"
I20260204 08:52:47.619969 1592 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 1 FOLLOWER]: Advancing to term 2
I20260204 08:52:47.626257 1592 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e719e457e94a5da71929720411ae69 in term 2.
I20260204 08:52:47.627041 1633 leader_election.cc:290] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [CANDIDATE]: Term 2 election: Requested vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005)
I20260204 08:52:47.627458 772 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1e719e457e94a5da71929720411ae69" candidate_term: 2 candidate_status { last_received { term: 1 index: 51 } } ignore_live_leader: false dest_uuid: "d083ecc9895345e5b06cd00ef0246c71"
I20260204 08:52:47.627763 817 leader_election.cc:304] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1e719e457e94a5da71929720411ae69, ebeeb5d4823b41b3a8e37546ff2f6e34; no voters:
I20260204 08:52:47.628546 1633 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 2 FOLLOWER]: Leader election won for term 2
I20260204 08:52:47.636891 1633 raft_consensus.cc:697] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 2 LEADER]: Becoming Leader. State: Replica: c1e719e457e94a5da71929720411ae69, State: Running, Role: LEADER
I20260204 08:52:47.637651 1633 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 51, Committed index: 51, Last appended: 1.51, Last appended by leader: 51, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:47.641379 1633 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1e719e457e94a5da71929720411ae69. Latest consensus state: current_term: 2 leader_uuid: "c1e719e457e94a5da71929720411ae69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:47.641780 1633 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [sys.catalog]: This master's current role is: LEADER
I20260204 08:52:47.643210 1637 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260204 08:52:47.645746 1341 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:52:47.650043 1637 catalog_manager.cc:679] Loaded metadata for table default.table_3956d1827061406b8d0a913345864f74 [id=105a9508f80c4190a40ab4ab66af2872]
I20260204 08:52:47.651522 1637 catalog_manager.cc:679] Loaded metadata for table default.table_7bcbc3cfbc41406cba3ed79125a46e82 [id=180667e663144b7ea79638946d6c76cc]
I20260204 08:52:47.652220 1637 catalog_manager.cc:679] Loaded metadata for table default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82]
I20260204 08:52:47.652845 1637 catalog_manager.cc:679] Loaded metadata for table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]
I20260204 08:52:47.653358 1637 catalog_manager.cc:679] Loaded metadata for table default.table_42998faad7dd446c8a35b76a3a9b5994 [id=2df47e721cd642e4b6fd5e61e172749c]
I20260204 08:52:47.653905 1637 catalog_manager.cc:679] Loaded metadata for table default.table_43ccd0dac1b24e20880a930959aa280a [id=31385bebafdd42fe89d79858c81a5a18]
I20260204 08:52:47.654412 1637 catalog_manager.cc:679] Loaded metadata for table default.table_73e876d60c444a5daa316e4bf7158614 [id=42a606fb50bf4c58be67f67afab90d53]
I20260204 08:52:47.654906 1637 catalog_manager.cc:679] Loaded metadata for table default.table_c97013dd16f84d8ba7cd528295f5ad49 [id=5f9f5b0850774ff9b8ecb93e141ff768]
I20260204 08:52:47.655432 1637 catalog_manager.cc:679] Loaded metadata for table default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807]
I20260204 08:52:47.655953 1637 catalog_manager.cc:679] Loaded metadata for table default.table_b4244312363c4346b110598eb01428a3 [id=7d4edb505f7940668b492a9d592a3e61]
I20260204 08:52:47.656479 1637 catalog_manager.cc:679] Loaded metadata for table default.table_798df4bc6a0640c4b26f8fe2b3444d6b [id=893f125488f745a5b0bc9ec3d07da00b]
I20260204 08:52:47.656975 1637 catalog_manager.cc:679] Loaded metadata for table default.table_9e950ede78084ef18f4e1051d9a42a90 [id=c626dc1c029e4b59902b4493f4e16a78]
I20260204 08:52:47.674135 1637 catalog_manager.cc:679] Loaded metadata for table default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294]
I20260204 08:52:47.677453 1637 tablet_loader.cc:96] loaded metadata for tablet 043cdafd4b11463bae674695e7a74854 (table default.table_798df4bc6a0640c4b26f8fe2b3444d6b [id=893f125488f745a5b0bc9ec3d07da00b])
I20260204 08:52:47.678424 1637 tablet_loader.cc:96] loaded metadata for tablet 2de648ce42af4a2fbd494c8cc3e38bb4 (table default.table_3956d1827061406b8d0a913345864f74 [id=105a9508f80c4190a40ab4ab66af2872])
I20260204 08:52:47.678966 1637 tablet_loader.cc:96] loaded metadata for tablet 35b597aac0264d36a44ae37c6ffb945e (table default.table_7bcbc3cfbc41406cba3ed79125a46e82 [id=180667e663144b7ea79638946d6c76cc])
I20260204 08:52:47.679895 1637 tablet_loader.cc:96] loaded metadata for tablet 5fe7a46289d84efcb7b1455bd317d279 (table default.table_b4244312363c4346b110598eb01428a3 [id=7d4edb505f7940668b492a9d592a3e61])
I20260204 08:52:47.680388 1637 tablet_loader.cc:96] loaded metadata for tablet 60e7158619d5432594db8192ca00efbb (table default.table_43ccd0dac1b24e20880a930959aa280a [id=31385bebafdd42fe89d79858c81a5a18])
I20260204 08:52:47.688858 1637 tablet_loader.cc:96] loaded metadata for tablet 7121dc10d79548169242f5b65331c08a (table default.table_42998faad7dd446c8a35b76a3a9b5994 [id=2df47e721cd642e4b6fd5e61e172749c])
I20260204 08:52:47.689690 1637 tablet_loader.cc:96] loaded metadata for tablet 80f72b647ee643afabcc5cafa9c43b13 (table default.table_9e950ede78084ef18f4e1051d9a42a90 [id=c626dc1c029e4b59902b4493f4e16a78])
I20260204 08:52:47.690169 1637 tablet_loader.cc:96] loaded metadata for tablet 81f37ac4f50a440d8e7292bfa05a0a7d (table default.table_73e876d60c444a5daa316e4bf7158614 [id=42a606fb50bf4c58be67f67afab90d53])
I20260204 08:52:47.690610 1637 tablet_loader.cc:96] loaded metadata for tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421])
I20260204 08:52:47.691007 1637 tablet_loader.cc:96] loaded metadata for tablet 9c96a74505f448e5ad1b8cc373ee82e4 (table default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807])
I20260204 08:52:47.691884 1637 tablet_loader.cc:96] loaded metadata for tablet bf91146e638f43e0b493a5732f6174be (table default.table_c97013dd16f84d8ba7cd528295f5ad49 [id=5f9f5b0850774ff9b8ecb93e141ff768])
I20260204 08:52:47.692684 1637 tablet_loader.cc:96] loaded metadata for tablet ebed8bb53e7947e08d0c960b13bacd8d (table default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294])
I20260204 08:52:47.693148 1637 tablet_loader.cc:96] loaded metadata for tablet fe0b2df895c643e7b69fb30a2b0b8e5c (table default.table_dd85a76e9cd94cc0b63f44ac7ed48a3a [id=19930831c99540688fb9a1e6ebfd8f82])
I20260204 08:52:47.693531 1637 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260204 08:52:47.695403 1637 catalog_manager.cc:1269] Loaded cluster ID: 41c7773c62174f70835c9cfa744e1148
I20260204 08:52:47.695585 1637 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260204 08:52:47.700266 1637 catalog_manager.cc:1514] Loading token signing keys...
I20260204 08:52:47.702948 1637 catalog_manager.cc:6038] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: Loaded TSK: 0
I20260204 08:52:47.706523 1637 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260204 08:52:47.708552 1592 raft_consensus.cc:1240] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 2 FOLLOWER]: Rejecting Update request from peer d083ecc9895345e5b06cd00ef0246c71 for earlier term 1. Current term is 2. Ops: [1.5-1.51]
I20260204 08:52:47.710400 1637 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260204 08:52:47.710736 1485 consensus_queue.cc:1059] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260204 08:52:47.711652 1468 consensus_queue.cc:1048] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:47.711784 842 raft_consensus.cc:1240] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [term 2 LEADER]: Rejecting Update request from peer d083ecc9895345e5b06cd00ef0246c71 for earlier term 1. Current term is 2. Ops: []
I20260204 08:52:47.714712 1468 consensus_queue.cc:1048] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:47.725162 1067 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:52:47.732944 1637 catalog_manager.cc:5876] Last processed Hive Metastore notification event ID: 11
I20260204 08:52:47.737779 1205 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:52:47.743088 1646 raft_consensus.cc:3055] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 1 LEADER]: Stepping down as leader of term 1
I20260204 08:52:47.743417 1646 raft_consensus.cc:740] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d083ecc9895345e5b06cd00ef0246c71, State: Running, Role: LEADER
I20260204 08:52:47.743932 1646 consensus_queue.cc:260] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 51, Committed index: 51, Last appended: 1.51, Last appended by leader: 51, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:47.756698 1342 heartbeater.cc:499] Master 127.0.104.188:38981 was elected leader, sending a full tablet report...
I20260204 08:52:47.761904 831 catalog_manager.cc:5654] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1234acd7d0db405085a5d95795007134 (127.0.104.131). New cstate: current_term: 1 leader_uuid: "1234acd7d0db405085a5d95795007134" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: HEALTHY } } }
I20260204 08:52:47.765996 831 catalog_manager.cc:5654] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1234acd7d0db405085a5d95795007134 (127.0.104.131). New cstate: current_term: 1 leader_uuid: "1234acd7d0db405085a5d95795007134" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: HEALTHY } } }
I20260204 08:52:47.767436 831 catalog_manager.cc:5746] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491) does not have the latest schema for tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
I20260204 08:52:47.769438 1068 heartbeater.cc:499] Master 127.0.104.188:38981 was elected leader, sending a full tablet report...
W20260204 08:52:47.776134 1645 consensus_queue.cc:1175] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c1e719e457e94a5da71929720411ae69" responder_term: 2 status { last_received { term: 2 index: 52 } last_committed_idx: 51 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer d083ecc9895345e5b06cd00ef0246c71 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260204 08:52:47.777375 1592 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 2 FOLLOWER]: Refusing update from remote peer c1e719e457e94a5da71929720411ae69: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 53. (index mismatch)
I20260204 08:52:47.778146 1633 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 52, Last known committed idx: 4, Time since last communication: 0.000s
W20260204 08:52:47.777586 747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.104.190:42471, user_credentials={real_user=slave}} blocked reactor thread for 67321.8us
I20260204 08:52:47.782799 1646 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 1 FOLLOWER]: Advancing to term 2
I20260204 08:52:47.783500 1468 raft_consensus.cc:493] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:47.783666 1468 raft_consensus.cc:515] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.784413 1468 leader_election.cc:290] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:47.785951 772 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 2 FOLLOWER]: Refusing update from remote peer c1e719e457e94a5da71929720411ae69: Log matching property violated. Preceding OpId in replica: term: 1 index: 51. Preceding OpId from leader: term: 2 index: 53. (index mismatch)
I20260204 08:52:47.785980 1022 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fe0b2df895c643e7b69fb30a2b0b8e5c" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:47.786214 1022 raft_consensus.cc:2468] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:47.786649 1232 leader_election.cc:304] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:47.786952 1457 raft_consensus.cc:2804] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:47.787076 1457 raft_consensus.cc:493] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:47.787190 1457 raft_consensus.cc:3060] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.787460 1633 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 52, Last known committed idx: 51, Time since last communication: 0.000s
I20260204 08:52:47.789894 1457 raft_consensus.cc:515] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.791036 1020 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fe0b2df895c643e7b69fb30a2b0b8e5c" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:47.791220 1457 leader_election.cc:290] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:47.791354 1020 raft_consensus.cc:3060] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.793032 1159 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fe0b2df895c643e7b69fb30a2b0b8e5c" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:47.793295 1159 raft_consensus.cc:2468] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:47.793795 1158 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fe0b2df895c643e7b69fb30a2b0b8e5c" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:47.793980 1158 raft_consensus.cc:3060] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:47.794799 1020 raft_consensus.cc:2468] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:47.796523 1232 leader_election.cc:304] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:47.797156 1538 raft_consensus.cc:2804] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:47.797336 1538 raft_consensus.cc:697] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Becoming Leader. State: Replica: 1234acd7d0db405085a5d95795007134, State: Running, Role: LEADER
I20260204 08:52:47.797628 1538 consensus_queue.cc:237] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:47.800506 1158 raft_consensus.cc:2468] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:47.802096 1646 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1e719e457e94a5da71929720411ae69. Latest consensus state: current_term: 2 leader_uuid: "c1e719e457e94a5da71929720411ae69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:47.802369 1646 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:47.804639 1653 mvcc.cc:204] Tried to move back new op lower bound from 7250719407205007360 to 7250719406649294848. Current Snapshot: MvccSnapshot[applied={T|T < 7250719407205007360}]
I20260204 08:52:47.805946 1656 mvcc.cc:204] Tried to move back new op lower bound from 7250719407205007360 to 7250719406649294848. Current Snapshot: MvccSnapshot[applied={T|T < 7250719407205007360}]
I20260204 08:52:47.806171 1206 heartbeater.cc:499] Master 127.0.104.188:38981 was elected leader, sending a full tablet report...
I20260204 08:52:47.818339 1646 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "c1e719e457e94a5da71929720411ae69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:47.818598 1646 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:47.818780 1636 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "c1e719e457e94a5da71929720411ae69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:47.819171 1636 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [sys.catalog]: This master's current role is: LEADER
I20260204 08:52:47.822724 832 catalog_manager.cc:5654] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129). New cstate: current_term: 1 leader_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: UNKNOWN } } }
I20260204 08:52:47.823468 832 catalog_manager.cc:5746] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675) does not have the latest schema for tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
I20260204 08:52:47.823742 832 catalog_manager.cc:5654] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129). New cstate: current_term: 1 leader_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: UNKNOWN } } }
I20260204 08:52:47.856977 830 catalog_manager.cc:5746] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737) does not have the latest schema for tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
I20260204 08:52:47.860613 830 catalog_manager.cc:5654] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 reported cstate change: term changed from 0 to 1, leader changed from <none> to fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "fdea56c2d1f5499087b939ec9f9dc070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: UNKNOWN } } }
I20260204 08:52:47.861620 830 catalog_manager.cc:5654] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 reported cstate change: term changed from 0 to 1, leader changed from <none> to fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "fdea56c2d1f5499087b939ec9f9dc070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: UNKNOWN } } }
I20260204 08:52:47.873844 829 catalog_manager.cc:5654] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1234acd7d0db405085a5d95795007134 (127.0.104.131). New cstate: current_term: 1 leader_uuid: "1234acd7d0db405085a5d95795007134" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: HEALTHY } } }
I20260204 08:52:47.875586 1160 raft_consensus.cc:1275] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Refusing update from remote peer 1234acd7d0db405085a5d95795007134: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260204 08:52:47.875777 1020 raft_consensus.cc:1275] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Refusing update from remote peer 1234acd7d0db405085a5d95795007134: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260204 08:52:47.875924 922 hms_client.cc:305] Time spent drop HMS table: real 1.149s user 0.000s sys 0.000s
I20260204 08:52:47.876274 753 catalog_manager.cc:2731] deleted HMS catalog entry for table default.table_43ccd0dac1b24e20880a930959aa280a [id=31385bebafdd42fe89d79858c81a5a18]
I20260204 08:52:47.876242 1457 consensus_queue.cc:1048] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:47.876838 1634 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1e719e457e94a5da71929720411ae69. Latest consensus state: current_term: 2 leader_uuid: "c1e719e457e94a5da71929720411ae69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:47.877123 1634 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:47.878460 1457 consensus_queue.cc:1048] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:47.882725 1636 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "c1e719e457e94a5da71929720411ae69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:47.882932 1636 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69 [sys.catalog]: This master's current role is: LEADER
I20260204 08:52:47.900475 1634 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "c1e719e457e94a5da71929720411ae69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:47.900730 1634 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:47.906754 830 catalog_manager.cc:5746] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675) does not have the latest schema for tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
I20260204 08:52:47.910154 756 catalog_manager.cc:2174] created HMS catalog entry for table [id=158d366d5ccf4b78b73903776a30cb35]
I20260204 08:52:47.910301 1650 mvcc.cc:204] Tried to move back new op lower bound from 7250719407609782272 to 7250719405385850880. Current Snapshot: MvccSnapshot[applied={T|T < 7250719407609782272}]
W20260204 08:52:47.911203 756 catalog_manager.cc:2202] Illegal state: an error occurred while writing to the sys-catalog: replica d083ecc9895345e5b06cd00ef0246c71 is not leader of this config: current role FOLLOWER
W20260204 08:52:47.918992 924 catalog_manager.cc:3922] Illegal state: an error occurred while updating the sys-catalog: replica d083ecc9895345e5b06cd00ef0246c71 is not leader of this config: current role FOLLOWER
W20260204 08:52:47.919335 924 hms_notification_log_listener.cc:330] Failed to handle Hive Metastore notification: 12 ALTER_TABLE default.table_f8273ef1a407421085918a8e3827f5f6: Illegal state: an error occurred while updating the sys-catalog: replica d083ecc9895345e5b06cd00ef0246c71 is not leader of this config: current role FOLLOWER
W20260204 08:52:47.919492 924 hms_notification_log_listener.cc:143] Hive Metastore notification log listener poll failed: Service unavailable: lost leadership while handling Hive Metastore notification log events: an error occurred while updating the sys-catalog: replica d083ecc9895345e5b06cd00ef0246c71 is not leader of this config: current role FOLLOWER
I20260204 08:52:47.919947 760 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:41804 (request call id 19) took 2698 ms. Trace:
I20260204 08:52:47.920058 760 rpcz_store.cc:276] 0204 08:52:45.221122 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:45.320212 (+ 99090us) service_pool.cc:225] Handling call
0204 08:52:46.135721 (+815509us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0204 08:52:47.919878 (+1784157us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":8840,"Hive Metastore.run_cpu_time_us":747,"Hive Metastore.run_wall_time_us":88610,"spinlock_wait_cycles":2816}
W20260204 08:52:47.921336 760 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.130:35287 (request call id 24) took 809 ms (client timeout 1000 ms). Trace:
W20260204 08:52:47.921445 760 rpcz_store.cc:269] 0204 08:52:47.112067 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.920498 (+808431us) service_pool.cc:225] Handling call
0204 08:52:47.921278 (+ 780us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260204 08:52:47.921725 1353 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.104.189:37005)
W20260204 08:52:47.922402 760 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.129:45029 (request call id 29) took 794 ms (client timeout 1000 ms). Trace:
W20260204 08:52:47.922487 760 rpcz_store.cc:269] 0204 08:52:47.128196 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.921677 (+793481us) service_pool.cc:225] Handling call
0204 08:52:47.922352 (+ 675us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260204 08:52:47.923322 760 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.131:34107 (request call id 22) took 770 ms (client timeout 1000 ms). Trace:
W20260204 08:52:47.923413 760 rpcz_store.cc:269] 0204 08:52:47.153031 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.922636 (+769605us) service_pool.cc:225] Handling call
0204 08:52:47.923263 (+ 627us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260204 08:52:47.926374 755 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:41804 (request call id 13) took 2817 ms. Trace:
I20260204 08:52:47.926503 755 rpcz_store.cc:276] 0204 08:52:45.108845 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:45.111586 (+ 2741us) service_pool.cc:225] Handling call
0204 08:52:46.135945 (+1024359us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0204 08:52:47.925766 (+1789821us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0204 08:52:47.926015 (+ 249us) catalog_manager.cc:3708] Apply alter schema
0204 08:52:47.926332 (+ 317us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":97383,"Hive Metastore.run_cpu_time_us":712,"Hive Metastore.run_wall_time_us":128119}
I20260204 08:52:47.927572 755 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41804 (request call id 35) took 2335 ms. Trace:
I20260204 08:52:47.927682 755 rpcz_store.cc:276] 0204 08:52:45.592167 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.926792 (+2334625us) service_pool.cc:225] Handling call
0204 08:52:47.927535 (+ 743us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260204 08:52:47.927695 830 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54400:
table { table_name: "default.Table_42998faad7dd446c8a35b76a3a9b5994" } new_table_name: "default.Table_f8273ef1a407421085918a8e3827f5f6" modify_external_catalogs: true
W20260204 08:52:47.931033 1353 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.104.188:38981)
I20260204 08:52:47.928244 755 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:41804 (request call id 36) took 2332 ms. Trace:
I20260204 08:52:47.932444 755 rpcz_store.cc:276] 0204 08:52:45.595435 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.927891 (+2332456us) service_pool.cc:225] Handling call
0204 08:52:47.928208 (+ 317us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260204 08:52:47.933178 755 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41804 (request call id 37) took 2294 ms. Trace:
I20260204 08:52:47.933301 755 rpcz_store.cc:276] 0204 08:52:45.639128 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.932745 (+2293617us) service_pool.cc:225] Handling call
0204 08:52:47.933139 (+ 394us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260204 08:52:47.933753 755 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41804 (request call id 38) took 2234 ms. Trace:
I20260204 08:52:47.933847 755 rpcz_store.cc:276] 0204 08:52:45.698945 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.933447 (+2234502us) service_pool.cc:225] Handling call
0204 08:52:47.933722 (+ 275us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260204 08:52:47.934268 755 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41804 (request call id 39) took 2172 ms. Trace:
I20260204 08:52:47.934363 755 rpcz_store.cc:276] 0204 08:52:45.761658 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.933979 (+2172321us) service_pool.cc:225] Handling call
0204 08:52:47.934240 (+ 261us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260204 08:52:47.934751 755 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41804 (request call id 40) took 2137 ms. Trace:
I20260204 08:52:47.934841 755 rpcz_store.cc:276] 0204 08:52:45.797487 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.934496 (+2137009us) service_pool.cc:225] Handling call
0204 08:52:47.934722 (+ 226us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260204 08:52:47.935277 755 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:41804 (request call id 41) took 1525 ms. Trace:
I20260204 08:52:47.935379 755 rpcz_store.cc:276] 0204 08:52:46.409573 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.934960 (+1525387us) service_pool.cc:225] Handling call
0204 08:52:47.935249 (+ 289us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260204 08:52:47.935788 755 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41804 (request call id 42) took 1160 ms. Trace:
I20260204 08:52:47.935878 755 rpcz_store.cc:276] 0204 08:52:46.775285 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.935522 (+1160237us) service_pool.cc:225] Handling call
0204 08:52:47.935758 (+ 236us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260204 08:52:47.935868 829 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54400:
name: "default.Table_97ec63740f2b4928af12e35b5eaa29cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:47.937160 1676 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260204 08:52:47.938668 1672 tablet.cc:1722] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:47.942047 1671 tablet.cc:1722] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:47.942894 832 catalog_manager.cc:5746] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675) does not have the latest schema for tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
I20260204 08:52:47.942909 1573 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" instance_seqno: 1770195163347042) as {username='slave'} at 127.0.104.129:41291; Asking this server to re-register.
I20260204 08:52:47.944144 1067 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:47.947127 1574 ts_manager.cc:194] Registered new tserver with Master: 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:47.947162 832 catalog_manager.cc:5746] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737) does not have the latest schema for tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
I20260204 08:52:47.950600 1573 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" instance_seqno: 1770195163874581) as {username='slave'} at 127.0.104.130:54429; Asking this server to re-register.
I20260204 08:52:47.951593 1205 heartbeater.cc:461] Registering TS with master...
W20260204 08:52:47.953992 1353 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.104.188:38981)
I20260204 08:52:47.956513 828 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54400:
name: "default.Table_477defa30c2c40e99360b505ef988a80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:47.957297 831 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54400:
table { table_name: "default.Table_b4244312363c4346b110598eb01428a3" } modify_external_catalogs: true
I20260204 08:52:47.957525 831 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54400:
table { table_name: "default.Table_b4244312363c4346b110598eb01428a3" } modify_external_catalogs: true
I20260204 08:52:47.964962 1574 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1234acd7d0db405085a5d95795007134" instance_seqno: 1770195164422899) as {username='slave'} at 127.0.104.131:44451; Asking this server to re-register.
I20260204 08:52:47.966084 1341 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:47.966487 1573 ts_manager.cc:194] Registered new tserver with Master: fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:47.969893 1574 ts_manager.cc:194] Registered new tserver with Master: 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:47.983372 753 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:41804 (request call id 24) took 2571 ms. Trace:
I20260204 08:52:47.983525 753 rpcz_store.cc:276] 0204 08:52:45.392526 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:45.608079 (+215553us) service_pool.cc:225] Handling call
0204 08:52:46.201844 (+593765us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0204 08:52:47.963519 (+1761675us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":525027,"Hive Metastore.run_cpu_time_us":407,"Hive Metastore.run_wall_time_us":1148959}
I20260204 08:52:47.985723 753 master_service.cc:954] ReplaceTablet: received request to replace tablet bf91146e638f43e0b493a5732f6174be from {username='slave'} at 127.0.0.1:41804
I20260204 08:52:47.985587 825 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54400:
name: "default.Table_f087faf331654fae99b16adcfb5f705f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:47.985752 824 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54400:
table { table_name: "default.Table_67b2871860f446a1a5551b16cbbcec4d" } new_table_name: "default.Table_52f9941ea7774cceafc55a0edbdb526c" modify_external_catalogs: true
W20260204 08:52:47.986454 759 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_27cce8dee4984465be213744abfadc9f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260204 08:52:47.986299 823 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54400:
name: "default.Table_780ed695e5324a469614b35852a643cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:47.987325 761 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:41804 (request call id 12) took 2912 ms. Trace:
I20260204 08:52:47.988214 827 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54400:
name: "default.Table_a638f2d4f2654854afdb0b065ee83f48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:47.987898 761 rpcz_store.cc:276] 0204 08:52:45.074874 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:45.074931 (+ 57us) service_pool.cc:225] Handling call
0204 08:52:46.135715 (+1060784us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0204 08:52:47.961508 (+1825793us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0204 08:52:47.987008 (+ 25500us) catalog_manager.cc:3708] Apply alter schema
0204 08:52:47.987288 (+ 280us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":312121,"Hive Metastore.run_cpu_time_us":716,"Hive Metastore.run_wall_time_us":75995,"mutex_wait_us":1004}
I20260204 08:52:47.990401 826 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54400:
name: "default.Table_1389d07a897b48439a706dce8abe591b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:47.990974 761 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41804 (request call id 45) took 1049 ms. Trace:
I20260204 08:52:47.992867 761 rpcz_store.cc:276] 0204 08:52:46.941287 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.990616 (+1049329us) service_pool.cc:225] Handling call
0204 08:52:47.990936 (+ 320us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260204 08:52:47.993139 761 master_service.cc:954] ReplaceTablet: received request to replace tablet 96ed25c622d946dfb4531ae603a80bad from {username='slave'} at 127.0.0.1:41804
I20260204 08:52:48.000146 1671 tablet.cc:1722] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:48.000200 754 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:41804 (request call id 23) took 2677 ms. Trace:
I20260204 08:52:48.004386 1672 tablet.cc:1722] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:48.009832 761 master_service.cc:954] ReplaceTablet: received request to replace tablet 9c96a74505f448e5ad1b8cc373ee82e4 from {username='slave'} at 127.0.0.1:41804
I20260204 08:52:48.020126 1680 tablet.cc:1722] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260204 08:52:48.020432 1353 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.104.188:38981)
I20260204 08:52:48.020565 1655 mvcc.cc:204] Tried to move back new op lower bound from 7250719407205007360 to 7250719406649294848. Current Snapshot: MvccSnapshot[applied={T|T < 7250719407205007360 or (T in {7250719407205007360})}]
I20260204 08:52:48.005028 754 rpcz_store.cc:276] 0204 08:52:45.311145 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:45.500673 (+189528us) service_pool.cc:225] Handling call
0204 08:52:46.136116 (+635443us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0204 08:52:47.961370 (+1825254us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0204 08:52:47.988150 (+ 26780us) catalog_manager.cc:3708] Apply alter schema
0204 08:52:47.988418 (+ 268us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":225391,"Hive Metastore.run_cpu_time_us":738,"Hive Metastore.run_wall_time_us":86366}
I20260204 08:52:48.022301 754 master_service.cc:954] ReplaceTablet: received request to replace tablet bf91146e638f43e0b493a5732f6174be from {username='slave'} at 127.0.0.1:41804
I20260204 08:52:48.024287 832 catalog_manager.cc:6199] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: default.table_f8273ef1a407421085918a8e3827f5f6 [id=2df47e721cd642e4b6fd5e61e172749c] alter complete (version 1)
I20260204 08:52:48.024410 754 master_service.cc:954] ReplaceTablet: received request to replace tablet 7121dc10d79548169242f5b65331c08a from {username='slave'} at 127.0.0.1:41804
I20260204 08:52:48.028527 753 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:41804 (request call id 44) took 1142 ms. Trace:
I20260204 08:52:48.029002 753 rpcz_store.cc:276] 0204 08:52:46.886347 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.985113 (+1098766us) service_pool.cc:225] Handling call
0204 08:52:48.005716 (+ 20603us) spinlock_profiling.cc:243] Waited 5.75 ms on lock 0x616000041254. stack: 00007f462b2aca86 00007f462b2ac840 00007f462a2873ad 0000000000984f94 0000000000984f5d 00007f463a6136bd 00007f463a6137b3 00007f46371da521 00007f463eea56fd 00007f463f03eadb 00007f4635b3d107 00007f4635b3ce35 00007f46322ac55e 00007f46322ab43e 00007f46322b1675 00007f46322b1f9f
0204 08:52:48.028482 (+ 22766us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":13227904}
I20260204 08:52:48.029829 755 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41804 (request call id 43) took 1108 ms. Trace:
I20260204 08:52:48.030615 1022 raft_consensus.cc:1275] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Refusing update from remote peer 1234acd7d0db405085a5d95795007134: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260204 08:52:48.031245 1468 consensus_queue.cc:1048] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:48.041429 1665 consensus_queue.cc:1048] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:48.045553 1158 raft_consensus.cc:1275] T 043cdafd4b11463bae674695e7a74854 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Refusing update from remote peer 1234acd7d0db405085a5d95795007134: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260204 08:52:48.046339 1468 consensus_queue.cc:1048] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:48.052095 1652 mvcc.cc:204] Tried to move back new op lower bound from 7250719408244588544 to 7250719404637798400. Current Snapshot: MvccSnapshot[applied={T|T < 7250719408244588544}]
I20260204 08:52:48.052258 755 rpcz_store.cc:276] 0204 08:52:46.828044 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.936000 (+1107956us) service_pool.cc:225] Handling call
0204 08:52:47.936271 (+ 271us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260204 08:52:48.052402 1680 tablet.cc:1722] T 043cdafd4b11463bae674695e7a74854 P fdea56c2d1f5499087b939ec9f9dc070: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:48.061527 888 catalog_manager.cc:2772] Deleting table default.table_43ccd0dac1b24e20880a930959aa280a [id=31385bebafdd42fe89d79858c81a5a18] in response to Hive Metastore notification log event 19
I20260204 08:52:48.068259 888 catalog_manager.cc:5941] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: Sending DeleteTablet for 3 replicas of tablet 60e7158619d5432594db8192ca00efbb
I20260204 08:52:48.070889 1680 tablet.cc:1722] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:48.070904 1672 tablet.cc:1722] T 043cdafd4b11463bae674695e7a74854 P 6d471268b9e643e5bbbf8bde1644a5ae: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:48.085770 1671 tablet.cc:1722] T 043cdafd4b11463bae674695e7a74854 P 1234acd7d0db405085a5d95795007134: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:48.093820 1160 raft_consensus.cc:1275] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Refusing update from remote peer 1234acd7d0db405085a5d95795007134: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260204 08:52:48.094666 1457 consensus_queue.cc:1048] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:48.107129 1276 tablet_service.cc:1552] Processing DeleteTablet for tablet 60e7158619d5432594db8192ca00efbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-04 08:52:48 UTC) from {username='slave'} at 127.0.0.1:36752
I20260204 08:52:48.107218 1671 tablet.cc:1722] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:48.107868 1680 tablet.cc:1722] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:48.115818 926 catalog_manager.cc:1612] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260204 08:52:48.117837 1665 consensus_queue.cc:1048] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260204 08:52:48.120285 749 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 96ed25c622d946dfb4531ae603a80bad consensus state has no leader
I20260204 08:52:48.123991 1021 raft_consensus.cc:1275] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Refusing update from remote peer 1234acd7d0db405085a5d95795007134: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260204 08:52:48.124857 1538 consensus_queue.cc:1048] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:48.133172 1672 tablet.cc:1722] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:48.137038 1423 consensus_queue.cc:1048] T 35b597aac0264d36a44ae37c6ffb945e P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:48.148725 1671 tablet.cc:1722] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:48.149394 1680 tablet.cc:1722] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:48.152226 1000 tablet_service.cc:1552] Processing DeleteTablet for tablet 60e7158619d5432594db8192ca00efbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-04 08:52:48 UTC) from {username='slave'} at 127.0.0.1:33412
I20260204 08:52:48.157477 1672 tablet.cc:1722] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:48.166085 1699 tablet_replica.cc:333] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:52:48.166354 1699 raft_consensus.cc:2243] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:52:48.172719 1693 tablet_replica.cc:333] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:52:48.172969 1693 raft_consensus.cc:2243] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:48.173228 1693 raft_consensus.cc:2272] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:48.173851 1693 ts_tablet_manager.cc:1916] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:48.176784 832 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54400:
name: "default.Table_4f900fd574c849aba171e6a64bc43dbd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:48.177271 830 catalog_manager.cc:5746] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675) does not have the latest schema for tablet 043cdafd4b11463bae674695e7a74854 (table default.table_9f93a995778f4fe6a4667cff0ef494b9 [id=893f125488f745a5b0bc9ec3d07da00b]). Expected version 1 got 0
I20260204 08:52:48.179294 830 catalog_manager.cc:5746] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737) does not have the latest schema for tablet 043cdafd4b11463bae674695e7a74854 (table default.table_9f93a995778f4fe6a4667cff0ef494b9 [id=893f125488f745a5b0bc9ec3d07da00b]). Expected version 1 got 0
I20260204 08:52:48.183712 1693 ts_tablet_manager.cc:1929] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260204 08:52:48.183903 1693 log.cc:1199] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/60e7158619d5432594db8192ca00efbb
I20260204 08:52:48.184322 1693 ts_tablet_manager.cc:1950] T 60e7158619d5432594db8192ca00efbb P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:52:48.192698 1699 pending_rounds.cc:70] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae: Trying to abort 1 pending ops.
I20260204 08:52:48.192935 1699 pending_rounds.cc:77] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7250719406124986368 op_type: NO_OP noop_request { }
I20260204 08:52:48.193113 1699 raft_consensus.cc:2889] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260204 08:52:48.193236 1699 raft_consensus.cc:2272] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:48.194003 1139 tablet_service.cc:1552] Processing DeleteTablet for tablet 60e7158619d5432594db8192ca00efbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-02-04 08:52:48 UTC) from {username='slave'} at 127.0.0.1:57288
I20260204 08:52:48.195416 1699 ts_tablet_manager.cc:1916] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:48.197046 830 catalog_manager.cc:6199] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: default.table_9f93a995778f4fe6a4667cff0ef494b9 [id=893f125488f745a5b0bc9ec3d07da00b] alter complete (version 1)
I20260204 08:52:48.207940 830 catalog_manager.cc:6199] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: default.table_e7fe420574dc40cdb70c4602f5a6e0db [id=19930831c99540688fb9a1e6ebfd8f82] alter complete (version 1)
I20260204 08:52:48.210363 1665 consensus_queue.cc:1048] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:48.213672 819 catalog_manager.cc:4985] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): tablet 60e7158619d5432594db8192ca00efbb (table default.table_43ccd0dac1b24e20880a930959aa280a [id=31385bebafdd42fe89d79858c81a5a18]) successfully deleted
I20260204 08:52:48.218098 1699 ts_tablet_manager.cc:1929] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:48.219667 1699 log.cc:1199] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/60e7158619d5432594db8192ca00efbb
I20260204 08:52:48.220448 1699 ts_tablet_manager.cc:1950] T 60e7158619d5432594db8192ca00efbb P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
W20260204 08:52:48.220860 829 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_97ec63740f2b4928af12e35b5eaa29cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260204 08:52:48.222558 828 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_477defa30c2c40e99360b505ef988a80 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260204 08:52:48.224607 823 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_780ed695e5324a469614b35852a643cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260204 08:52:48.225716 827 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_a638f2d4f2654854afdb0b065ee83f48 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260204 08:52:48.226094 1700 tablet_replica.cc:333] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:52:48.226318 1700 raft_consensus.cc:2243] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:48.226540 1700 raft_consensus.cc:2272] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260204 08:52:48.226665 826 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1389d07a897b48439a706dce8abe591b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260204 08:52:48.227140 1700 ts_tablet_manager.cc:1916] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260204 08:52:48.227715 825 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f087faf331654fae99b16adcfb5f705f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260204 08:52:48.229086 819 catalog_manager.cc:4985] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): tablet 60e7158619d5432594db8192ca00efbb (table default.table_43ccd0dac1b24e20880a930959aa280a [id=31385bebafdd42fe89d79858c81a5a18]) successfully deleted
I20260204 08:52:48.231683 830 catalog_manager.cc:6199] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: default.table_f388b768c4b14290a90dd29962cc3fc4 [id=105a9508f80c4190a40ab4ab66af2872] alter complete (version 1)
I20260204 08:52:48.246625 830 master_service.cc:954] ReplaceTablet: received request to replace tablet 96ed25c622d946dfb4531ae603a80bad from {username='slave'} at 127.0.0.1:54400
I20260204 08:52:48.257884 1700 ts_tablet_manager.cc:1929] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260204 08:52:48.258177 1700 log.cc:1199] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/60e7158619d5432594db8192ca00efbb
I20260204 08:52:48.258644 1700 ts_tablet_manager.cc:1950] T 60e7158619d5432594db8192ca00efbb P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:52:48.259836 819 catalog_manager.cc:4985] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737): tablet 60e7158619d5432594db8192ca00efbb (table default.table_43ccd0dac1b24e20880a930959aa280a [id=31385bebafdd42fe89d79858c81a5a18]) successfully deleted
I20260204 08:52:48.260725 830 catalog_manager.cc:5941] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: Sending DeleteTablet for 3 replicas of tablet 96ed25c622d946dfb4531ae603a80bad
I20260204 08:52:48.261315 830 catalog_manager.cc:6724] ReplaceTablet: tablet 96ed25c622d946dfb4531ae603a80bad deleted and replaced by tablet c9ce4ab8ce1f48868b890df81e551eed
I20260204 08:52:48.261646 830 master_service.cc:954] ReplaceTablet: received request to replace tablet 9c96a74505f448e5ad1b8cc373ee82e4 from {username='slave'} at 127.0.0.1:54400
I20260204 08:52:48.261675 1139 tablet_service.cc:1552] Processing DeleteTablet for tablet 96ed25c622d946dfb4531ae603a80bad with delete_type TABLET_DATA_DELETED (replaced by tablet c9ce4ab8ce1f48868b890df81e551eed) from {username='slave'} at 127.0.0.1:57288
I20260204 08:52:48.261729 1000 tablet_service.cc:1552] Processing DeleteTablet for tablet 96ed25c622d946dfb4531ae603a80bad with delete_type TABLET_DATA_DELETED (replaced by tablet c9ce4ab8ce1f48868b890df81e551eed) from {username='slave'} at 127.0.0.1:33412
I20260204 08:52:48.262010 1699 tablet_replica.cc:333] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:52:48.262221 1699 raft_consensus.cc:2243] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:48.262447 1699 raft_consensus.cc:2272] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:48.263381 1276 tablet_service.cc:1552] Processing DeleteTablet for tablet 96ed25c622d946dfb4531ae603a80bad with delete_type TABLET_DATA_DELETED (replaced by tablet c9ce4ab8ce1f48868b890df81e551eed) from {username='slave'} at 127.0.0.1:36752
I20260204 08:52:48.264076 1699 ts_tablet_manager.cc:1916] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:48.264505 1693 tablet_replica.cc:333] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:52:48.264727 1693 raft_consensus.cc:2243] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:52:48.265085 1693 raft_consensus.cc:2272] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:48.265290 1700 tablet_replica.cc:333] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:52:48.265501 1700 raft_consensus.cc:2243] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:48.265813 1700 raft_consensus.cc:2272] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:48.265894 1693 ts_tablet_manager.cc:1916] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:48.267021 1700 ts_tablet_manager.cc:1916] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:48.272653 1693 ts_tablet_manager.cc:1929] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260204 08:52:48.272959 1693 log.cc:1199] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/96ed25c622d946dfb4531ae603a80bad
I20260204 08:52:48.274289 830 catalog_manager.cc:5941] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: Sending DeleteTablet for 3 replicas of tablet 9c96a74505f448e5ad1b8cc373ee82e4
I20260204 08:52:48.274839 830 catalog_manager.cc:6724] ReplaceTablet: tablet 9c96a74505f448e5ad1b8cc373ee82e4 deleted and replaced by tablet d3c5509788ad4f8c858db788681b0920
I20260204 08:52:48.276149 1693 ts_tablet_manager.cc:1950] T 96ed25c622d946dfb4531ae603a80bad P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:52:48.276546 1139 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c96a74505f448e5ad1b8cc373ee82e4 with delete_type TABLET_DATA_DELETED (replaced by tablet d3c5509788ad4f8c858db788681b0920) from {username='slave'} at 127.0.0.1:57288
I20260204 08:52:48.276957 1002 tablet_service.cc:1505] Processing CreateTablet for tablet c9ce4ab8ce1f48868b890df81e551eed (DEFAULT_TABLE table=default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:48.277416 1002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ce4ab8ce1f48868b890df81e551eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:48.277406 1139 tablet_service.cc:1505] Processing CreateTablet for tablet c9ce4ab8ce1f48868b890df81e551eed (DEFAULT_TABLE table=default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:48.277783 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ce4ab8ce1f48868b890df81e551eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:48.280511 1000 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c96a74505f448e5ad1b8cc373ee82e4 with delete_type TABLET_DATA_DELETED (replaced by tablet d3c5509788ad4f8c858db788681b0920) from {username='slave'} at 127.0.0.1:33412
I20260204 08:52:48.281167 1276 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c96a74505f448e5ad1b8cc373ee82e4 with delete_type TABLET_DATA_DELETED (replaced by tablet d3c5509788ad4f8c858db788681b0920) from {username='slave'} at 127.0.0.1:36752
I20260204 08:52:48.281307 819 catalog_manager.cc:4985] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]) successfully deleted
I20260204 08:52:48.281670 1275 tablet_service.cc:1505] Processing CreateTablet for tablet c9ce4ab8ce1f48868b890df81e551eed (DEFAULT_TABLE table=default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:48.282099 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ce4ab8ce1f48868b890df81e551eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:48.282232 1693 tablet_replica.cc:333] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:52:48.282452 1693 raft_consensus.cc:2243] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:52:48.282804 1693 raft_consensus.cc:2272] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:48.284819 1693 ts_tablet_manager.cc:1916] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:48.290318 830 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54400:
table { table_name: "default.Table_c97013dd16f84d8ba7cd528295f5ad49" } new_table_name: "default.Table_b3a5ec0b6897403bb998ef249c195d5f" modify_external_catalogs: true
I20260204 08:52:48.292798 1704 tablet_bootstrap.cc:492] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:48.295560 1704 tablet_bootstrap.cc:654] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:48.297703 1000 tablet_service.cc:1505] Processing CreateTablet for tablet d3c5509788ad4f8c858db788681b0920 (DEFAULT_TABLE table=default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:48.298192 1000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c5509788ad4f8c858db788681b0920. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:48.297284 1700 ts_tablet_manager.cc:1929] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260204 08:52:48.298343 1700 log.cc:1199] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/96ed25c622d946dfb4531ae603a80bad
I20260204 08:52:48.298738 1700 ts_tablet_manager.cc:1950] T 96ed25c622d946dfb4531ae603a80bad P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:52:48.299764 819 catalog_manager.cc:4985] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737): tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]) successfully deleted
I20260204 08:52:48.301084 1700 tablet_replica.cc:333] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:52:48.301276 1700 raft_consensus.cc:2243] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:48.301476 1700 raft_consensus.cc:2272] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:48.302559 1699 ts_tablet_manager.cc:1929] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260204 08:52:48.302723 1699 log.cc:1199] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/96ed25c622d946dfb4531ae603a80bad
I20260204 08:52:48.303870 1699 ts_tablet_manager.cc:1950] T 96ed25c622d946dfb4531ae603a80bad P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:52:48.303689 1139 tablet_service.cc:1505] Processing CreateTablet for tablet d3c5509788ad4f8c858db788681b0920 (DEFAULT_TABLE table=default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:48.305467 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c5509788ad4f8c858db788681b0920. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:48.305608 1276 tablet_service.cc:1505] Processing CreateTablet for tablet d3c5509788ad4f8c858db788681b0920 (DEFAULT_TABLE table=default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:48.306048 1276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c5509788ad4f8c858db788681b0920. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:48.312602 1700 ts_tablet_manager.cc:1916] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:48.314460 1693 ts_tablet_manager.cc:1929] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:48.314755 1693 log.cc:1199] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/9c96a74505f448e5ad1b8cc373ee82e4
I20260204 08:52:48.315302 1693 ts_tablet_manager.cc:1950] T 9c96a74505f448e5ad1b8cc373ee82e4 P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:52:48.316529 819 catalog_manager.cc:4985] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): tablet 9c96a74505f448e5ad1b8cc373ee82e4 (table default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807]) successfully deleted
I20260204 08:52:48.317481 1699 tablet_replica.cc:333] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:52:48.317724 1699 raft_consensus.cc:2243] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:48.317884 1699 raft_consensus.cc:2272] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:48.317654 819 catalog_manager.cc:4985] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]) successfully deleted
I20260204 08:52:48.320983 1707 tablet_bootstrap.cc:492] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:48.322114 1699 ts_tablet_manager.cc:1916] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:48.322223 1704 tablet_bootstrap.cc:492] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:48.322413 1704 ts_tablet_manager.cc:1403] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.000s
I20260204 08:52:48.323400 1704 raft_consensus.cc:359] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:48.323762 1704 raft_consensus.cc:385] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:48.323921 1704 raft_consensus.cc:740] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:48.324118 1707 tablet_bootstrap.cc:654] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:48.324395 1704 consensus_queue.cc:260] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:48.325557 1705 tablet_bootstrap.cc:492] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:48.329614 1705 tablet_bootstrap.cc:654] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:48.329614 1707 tablet_bootstrap.cc:492] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:48.329783 1707 ts_tablet_manager.cc:1403] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.002s
I20260204 08:52:48.329792 1704 ts_tablet_manager.cc:1434] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260204 08:52:48.330780 1699 ts_tablet_manager.cc:1929] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:48.330687 1707 raft_consensus.cc:359] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:48.330930 1699 log.cc:1199] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/9c96a74505f448e5ad1b8cc373ee82e4
I20260204 08:52:48.330951 1707 raft_consensus.cc:385] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:48.331092 1707 raft_consensus.cc:740] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:48.331351 1699 ts_tablet_manager.cc:1950] T 9c96a74505f448e5ad1b8cc373ee82e4 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:52:48.331373 1707 consensus_queue.cc:260] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:48.333590 1707 ts_tablet_manager.cc:1434] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260204 08:52:48.333954 1707 tablet_bootstrap.cc:492] T d3c5509788ad4f8c858db788681b0920 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:48.334841 1704 tablet_bootstrap.cc:492] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:48.336712 1707 tablet_bootstrap.cc:654] T d3c5509788ad4f8c858db788681b0920 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:48.338024 1704 tablet_bootstrap.cc:654] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:48.340401 1700 ts_tablet_manager.cc:1929] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:48.340575 1700 log.cc:1199] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/9c96a74505f448e5ad1b8cc373ee82e4
I20260204 08:52:48.341039 1700 ts_tablet_manager.cc:1950] T 9c96a74505f448e5ad1b8cc373ee82e4 P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:52:48.342170 819 catalog_manager.cc:4985] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737): tablet 9c96a74505f448e5ad1b8cc373ee82e4 (table default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807]) successfully deleted
I20260204 08:52:48.343482 1704 tablet_bootstrap.cc:492] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:48.343688 1704 ts_tablet_manager.cc:1403] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260204 08:52:48.344595 1704 raft_consensus.cc:359] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:48.344825 1704 raft_consensus.cc:385] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:48.344924 1704 raft_consensus.cc:740] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:48.345216 819 catalog_manager.cc:4985] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): tablet 9c96a74505f448e5ad1b8cc373ee82e4 (table default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807]) successfully deleted
I20260204 08:52:48.345188 1704 consensus_queue.cc:260] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:48.345713 1704 ts_tablet_manager.cc:1434] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260204 08:52:48.347349 1707 tablet_bootstrap.cc:492] T d3c5509788ad4f8c858db788681b0920 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:48.347589 1707 ts_tablet_manager.cc:1403] T d3c5509788ad4f8c858db788681b0920 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.003s
I20260204 08:52:48.348421 1707 raft_consensus.cc:359] T d3c5509788ad4f8c858db788681b0920 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:48.348623 1707 raft_consensus.cc:385] T d3c5509788ad4f8c858db788681b0920 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:48.348698 1707 raft_consensus.cc:740] T d3c5509788ad4f8c858db788681b0920 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:48.348920 1707 consensus_queue.cc:260] T d3c5509788ad4f8c858db788681b0920 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:48.350314 1707 ts_tablet_manager.cc:1434] T d3c5509788ad4f8c858db788681b0920 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260204 08:52:48.350486 756 catalog_manager.cc:2184] deleted orphaned HMS catalog entry for table [id=158d366d5ccf4b78b73903776a30cb35]
I20260204 08:52:48.350963 756 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41804 (request call id 25) took 2929 ms. Trace:
I20260204 08:52:48.351089 756 rpcz_store.cc:276] 0204 08:52:45.421371 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:45.612796 (+191425us) service_pool.cc:225] Handling call
0204 08:52:46.203302 (+590506us) catalog_manager.cc:2101] Acquired catalog manager lock
0204 08:52:46.203806 (+ 504us) catalog_manager.cc:2147] Created new table and tablet info
0204 08:52:47.910112 (+1706306us) catalog_manager.cc:2173] Created new table in HMS catalog
0204 08:52:47.911011 (+ 899us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:47.911015 (+ 4us) write_op.cc:454] Released schema lock
0204 08:52:47.911323 (+ 308us) catalog_manager.cc:2181] Rolling back HMS table creation
0204 08:52:48.350907 (+439584us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:47.910912 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica d083ecc9895345e5b06cd00ef0246c71 is not leader of this config: current role FOLLOWER)
0204 08:52:47.910931 (+ 19us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:47.910938 (+ 7us) write_op.cc:454] Released schema lock
0204 08:52:47.910943 (+ 5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1676929,"Hive Metastore.run_cpu_time_us":549,"Hive Metastore.run_wall_time_us":467967,"child_traces":[["op",{}]]}
I20260204 08:52:48.358878 1705 tablet_bootstrap.cc:492] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:48.359355 1705 ts_tablet_manager.cc:1403] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.009s
I20260204 08:52:48.360205 1705 raft_consensus.cc:359] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:48.360458 1705 raft_consensus.cc:385] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:48.360594 1705 raft_consensus.cc:740] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:48.360879 1705 consensus_queue.cc:260] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:48.361480 1705 ts_tablet_manager.cc:1434] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260204 08:52:48.361793 1705 tablet_bootstrap.cc:492] T d3c5509788ad4f8c858db788681b0920 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:48.368021 1705 tablet_bootstrap.cc:654] T d3c5509788ad4f8c858db788681b0920 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:48.381368 1705 tablet_bootstrap.cc:492] T d3c5509788ad4f8c858db788681b0920 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:48.381603 1705 ts_tablet_manager.cc:1403] T d3c5509788ad4f8c858db788681b0920 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20260204 08:52:48.382555 1705 raft_consensus.cc:359] T d3c5509788ad4f8c858db788681b0920 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:48.382804 1705 raft_consensus.cc:385] T d3c5509788ad4f8c858db788681b0920 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:48.382929 1705 raft_consensus.cc:740] T d3c5509788ad4f8c858db788681b0920 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:48.383216 1705 consensus_queue.cc:260] T d3c5509788ad4f8c858db788681b0920 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:48.384874 1705 ts_tablet_manager.cc:1434] T d3c5509788ad4f8c858db788681b0920 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260204 08:52:48.389184 1665 raft_consensus.cc:493] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:48.389642 1667 raft_consensus.cc:493] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:48.389799 1667 raft_consensus.cc:515] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:48.389544 1665 raft_consensus.cc:515] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:48.390560 1667 leader_election.cc:290] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:48.390723 1665 leader_election.cc:290] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:48.391144 1020 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d3c5509788ad4f8c858db788681b0920" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:48.391377 1020 raft_consensus.cc:2468] T d3c5509788ad4f8c858db788681b0920 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:48.391388 1295 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d3c5509788ad4f8c858db788681b0920" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:48.391599 1295 raft_consensus.cc:2468] T d3c5509788ad4f8c858db788681b0920 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:48.391762 1019 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c9ce4ab8ce1f48868b890df81e551eed" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:48.391944 1019 raft_consensus.cc:2468] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:48.392176 1096 leader_election.cc:304] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:48.392236 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c9ce4ab8ce1f48868b890df81e551eed" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:48.392515 1665 raft_consensus.cc:2804] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:48.392501 1296 raft_consensus.cc:2468] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:48.392588 1665 raft_consensus.cc:493] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:48.392673 1665 raft_consensus.cc:3060] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:48.394153 1096 leader_election.cc:304] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:48.394655 1667 raft_consensus.cc:2804] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:48.394765 1667 raft_consensus.cc:493] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:48.394845 1667 raft_consensus.cc:3060] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:48.396432 1665 raft_consensus.cc:515] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:48.397112 1665 leader_election.cc:290] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:48.397506 1667 raft_consensus.cc:515] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:48.397994 1667 leader_election.cc:290] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:48.399029 1295 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c9ce4ab8ce1f48868b890df81e551eed" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:48.398983 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d3c5509788ad4f8c858db788681b0920" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:48.399256 1296 raft_consensus.cc:3060] T d3c5509788ad4f8c858db788681b0920 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:48.399356 1295 raft_consensus.cc:3060] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:48.402200 1295 raft_consensus.cc:2468] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:48.402240 1296 raft_consensus.cc:2468] T d3c5509788ad4f8c858db788681b0920 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:48.403338 1019 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d3c5509788ad4f8c858db788681b0920" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:48.403561 1019 raft_consensus.cc:3060] T d3c5509788ad4f8c858db788681b0920 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:48.405898 1019 raft_consensus.cc:2468] T d3c5509788ad4f8c858db788681b0920 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:48.406553 1020 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c9ce4ab8ce1f48868b890df81e551eed" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:48.406770 1020 raft_consensus.cc:3060] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:48.408933 1020 raft_consensus.cc:2468] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:48.409446 1096 leader_election.cc:304] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:48.409770 1667 raft_consensus.cc:2804] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:48.409818 1096 leader_election.cc:304] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:48.410087 1665 raft_consensus.cc:2804] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:48.410398 1665 raft_consensus.cc:697] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Becoming Leader. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Running, Role: LEADER
I20260204 08:52:48.410744 1665 consensus_queue.cc:237] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:48.411202 1667 raft_consensus.cc:697] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Becoming Leader. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Running, Role: LEADER
I20260204 08:52:48.411473 1667 consensus_queue.cc:237] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:48.423208 829 catalog_manager.cc:2174] created HMS catalog entry for table [id=2a58ed39876b46aba637d263234e55d9]
I20260204 08:52:48.432785 829 catalog_manager.cc:5746] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737) does not have the latest schema for tablet c9ce4ab8ce1f48868b890df81e551eed (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
W20260204 08:52:48.433447 829 catalog_manager.cc:5846] Failed to send AlterTable RPC for tablet c9ce4ab8ce1f48868b890df81e551eed (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet c9ce4ab8ce1f48868b890df81e551eed consensus state has no leader
I20260204 08:52:48.434517 829 catalog_manager.cc:5746] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491) does not have the latest schema for tablet c9ce4ab8ce1f48868b890df81e551eed (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
I20260204 08:52:48.450444 1002 tablet_service.cc:1505] Processing CreateTablet for tablet cb72d1554d9b4c33b6cd7c01fad765db (DEFAULT_TABLE table=default.table_97ec63740f2b4928af12e35b5eaa29cd [id=2a58ed39876b46aba637d263234e55d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:48.450462 1139 tablet_service.cc:1505] Processing CreateTablet for tablet cb72d1554d9b4c33b6cd7c01fad765db (DEFAULT_TABLE table=default.table_97ec63740f2b4928af12e35b5eaa29cd [id=2a58ed39876b46aba637d263234e55d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:48.450910 1002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb72d1554d9b4c33b6cd7c01fad765db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:48.450912 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb72d1554d9b4c33b6cd7c01fad765db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:48.455530 1704 tablet_bootstrap.cc:492] T cb72d1554d9b4c33b6cd7c01fad765db P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:48.455935 1275 tablet_service.cc:1505] Processing CreateTablet for tablet cb72d1554d9b4c33b6cd7c01fad765db (DEFAULT_TABLE table=default.table_97ec63740f2b4928af12e35b5eaa29cd [id=2a58ed39876b46aba637d263234e55d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:48.456734 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb72d1554d9b4c33b6cd7c01fad765db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:48.457297 1704 tablet_bootstrap.cc:654] T cb72d1554d9b4c33b6cd7c01fad765db P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:48.457556 1705 tablet_bootstrap.cc:492] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:48.459762 1705 tablet_bootstrap.cc:654] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:48.472608 829 catalog_manager.cc:5746] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675) does not have the latest schema for tablet c9ce4ab8ce1f48868b890df81e551eed (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
I20260204 08:52:48.473340 1705 tablet_bootstrap.cc:492] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:48.473598 1705 ts_tablet_manager.cc:1403] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.008s
I20260204 08:52:48.474427 1705 raft_consensus.cc:359] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:48.474721 1705 raft_consensus.cc:385] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:48.474845 1705 raft_consensus.cc:740] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:48.475198 1705 consensus_queue.cc:260] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:48.479873 1707 tablet_bootstrap.cc:492] T cb72d1554d9b4c33b6cd7c01fad765db P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:48.479688 829 catalog_manager.cc:5654] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 reported cstate change: term changed from 0 to 1, leader changed from <none> to fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "fdea56c2d1f5499087b939ec9f9dc070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: HEALTHY } } }
I20260204 08:52:48.480048 829 catalog_manager.cc:5746] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737) does not have the latest schema for tablet c9ce4ab8ce1f48868b890df81e551eed (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
I20260204 08:52:48.480355 829 catalog_manager.cc:5654] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 reported cstate change: term changed from 0 to 1, leader changed from <none> to fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "fdea56c2d1f5499087b939ec9f9dc070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: UNKNOWN } } }
I20260204 08:52:48.480741 1704 tablet_bootstrap.cc:492] T cb72d1554d9b4c33b6cd7c01fad765db P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:48.480913 1704 ts_tablet_manager.cc:1403] T cb72d1554d9b4c33b6cd7c01fad765db P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.025s user 0.011s sys 0.010s
W20260204 08:52:48.481459 1713 log.cc:927] Time spent T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070: Append to log took a long time: real 0.069s user 0.000s sys 0.000s
I20260204 08:52:48.481793 1704 raft_consensus.cc:359] T cb72d1554d9b4c33b6cd7c01fad765db P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:48.482007 1704 raft_consensus.cc:385] T cb72d1554d9b4c33b6cd7c01fad765db P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:48.482088 1704 raft_consensus.cc:740] T cb72d1554d9b4c33b6cd7c01fad765db P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:48.482208 1707 tablet_bootstrap.cc:654] T cb72d1554d9b4c33b6cd7c01fad765db P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:48.482326 1704 consensus_queue.cc:260] T cb72d1554d9b4c33b6cd7c01fad765db P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:48.482823 1704 ts_tablet_manager.cc:1434] T cb72d1554d9b4c33b6cd7c01fad765db P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
W20260204 08:52:48.486356 816 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet c9ce4ab8ce1f48868b890df81e551eed (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet c9ce4ab8ce1f48868b890df81e551eed consensus state has no leader
I20260204 08:52:48.490386 1705 ts_tablet_manager.cc:1434] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.017s user 0.002s sys 0.000s
I20260204 08:52:48.497874 1606 consensus_queue.cc:1048] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:48.502725 829 catalog_manager.cc:5746] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491) does not have the latest schema for tablet c9ce4ab8ce1f48868b890df81e551eed (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
I20260204 08:52:48.520268 1707 tablet_bootstrap.cc:492] T cb72d1554d9b4c33b6cd7c01fad765db P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:48.520519 1707 ts_tablet_manager.cc:1403] T cb72d1554d9b4c33b6cd7c01fad765db P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.010s
I20260204 08:52:48.522310 1707 raft_consensus.cc:359] T cb72d1554d9b4c33b6cd7c01fad765db P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:48.522547 1707 raft_consensus.cc:385] T cb72d1554d9b4c33b6cd7c01fad765db P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:48.522646 1707 raft_consensus.cc:740] T cb72d1554d9b4c33b6cd7c01fad765db P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:48.523942 1707 consensus_queue.cc:260] T cb72d1554d9b4c33b6cd7c01fad765db P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:48.525635 1707 ts_tablet_manager.cc:1434] T cb72d1554d9b4c33b6cd7c01fad765db P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260204 08:52:48.533504 829 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54400:
table { table_name: "default.Table_9f93a995778f4fe6a4667cff0ef494b9" } new_table_name: "default.Table_b4e59736cf4b48b4b920d4ad64d8c522" modify_external_catalogs: true
I20260204 08:52:48.571527 1020 raft_consensus.cc:1275] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Refusing update from remote peer fdea56c2d1f5499087b939ec9f9dc070: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260204 08:52:48.572178 1719 consensus_queue.cc:1048] T ebed8bb53e7947e08d0c960b13bacd8d P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:48.572744 1296 raft_consensus.cc:1275] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Refusing update from remote peer fdea56c2d1f5499087b939ec9f9dc070: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260204 08:52:48.573745 1665 consensus_queue.cc:1048] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:48.578064 1665 consensus_queue.cc:1048] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:48.583348 828 catalog_manager.cc:2174] created HMS catalog entry for table [id=b1e79546138b4dcb9c5a68977af78bf0]
I20260204 08:52:48.593138 828 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54400:
table { table_name: "default.Table_f388b768c4b14290a90dd29962cc3fc4" } modify_external_catalogs: true
I20260204 08:52:48.593425 828 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54400:
table { table_name: "default.Table_f388b768c4b14290a90dd29962cc3fc4" } modify_external_catalogs: true
I20260204 08:52:48.600778 1680 tablet.cc:1722] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:48.601676 1672 tablet.cc:1722] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:48.603080 1671 tablet.cc:1722] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260204 08:52:48.610464 762 catalog_manager.cc:2731] deleted HMS catalog entry for table default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294]
I20260204 08:52:48.611179 762 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:41804 (request call id 26) took 3173 ms. Trace:
I20260204 08:52:48.611331 762 rpcz_store.cc:276] 0204 08:52:45.437990 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:46.269716 (+831726us) service_pool.cc:225] Handling call
0204 08:52:47.925924 (+1656208us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0204 08:52:48.611136 (+685212us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":424186,"Hive Metastore.run_cpu_time_us":214,"Hive Metastore.run_wall_time_us":259913}
I20260204 08:52:48.616694 1002 tablet_service.cc:1505] Processing CreateTablet for tablet 4c26152dcc2147d6b0cb8d1242a7a73b (DEFAULT_TABLE table=default.table_477defa30c2c40e99360b505ef988a80 [id=b1e79546138b4dcb9c5a68977af78bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:48.617192 1002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c26152dcc2147d6b0cb8d1242a7a73b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:48.621505 1139 tablet_service.cc:1505] Processing CreateTablet for tablet 4c26152dcc2147d6b0cb8d1242a7a73b (DEFAULT_TABLE table=default.table_477defa30c2c40e99360b505ef988a80 [id=b1e79546138b4dcb9c5a68977af78bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:48.621976 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c26152dcc2147d6b0cb8d1242a7a73b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:48.622176 1275 tablet_service.cc:1505] Processing CreateTablet for tablet 4c26152dcc2147d6b0cb8d1242a7a73b (DEFAULT_TABLE table=default.table_477defa30c2c40e99360b505ef988a80 [id=b1e79546138b4dcb9c5a68977af78bf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:48.622602 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c26152dcc2147d6b0cb8d1242a7a73b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:48.628130 1704 tablet_bootstrap.cc:492] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:48.629927 1704 tablet_bootstrap.cc:654] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:48.631057 1705 tablet_bootstrap.cc:492] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:48.631290 1707 tablet_bootstrap.cc:492] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:48.632828 1705 tablet_bootstrap.cc:654] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:48.633000 1707 tablet_bootstrap.cc:654] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:48.637748 1705 tablet_bootstrap.cc:492] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:48.638027 1705 ts_tablet_manager.cc:1403] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.002s
I20260204 08:52:48.638892 1704 tablet_bootstrap.cc:492] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:48.639070 1704 ts_tablet_manager.cc:1403] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260204 08:52:48.639539 1705 raft_consensus.cc:359] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:48.639842 1705 raft_consensus.cc:385] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:48.639824 1704 raft_consensus.cc:359] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:48.640007 1704 raft_consensus.cc:385] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:48.640080 1704 raft_consensus.cc:740] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:48.640106 1705 raft_consensus.cc:740] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:48.640359 1704 consensus_queue.cc:260] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:48.640492 1705 consensus_queue.cc:260] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:48.643721 1704 ts_tablet_manager.cc:1434] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260204 08:52:48.644625 1705 ts_tablet_manager.cc:1434] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260204 08:52:48.664963 1707 tablet_bootstrap.cc:492] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:48.665207 1707 ts_tablet_manager.cc:1403] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.034s user 0.004s sys 0.001s
I20260204 08:52:48.666113 1707 raft_consensus.cc:359] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:48.666348 1707 raft_consensus.cc:385] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:48.666428 1707 raft_consensus.cc:740] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:48.666700 1707 consensus_queue.cc:260] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:48.667256 1707 ts_tablet_manager.cc:1434] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260204 08:52:48.865243 758 catalog_manager.cc:2731] deleted HMS catalog entry for table default.table_73e876d60c444a5daa316e4bf7158614 [id=42a606fb50bf4c58be67f67afab90d53]
I20260204 08:52:48.865845 758 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:41804 (request call id 28) took 3362 ms. Trace:
I20260204 08:52:48.865976 758 rpcz_store.cc:276] 0204 08:52:45.503072 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:46.670314 (+1167242us) service_pool.cc:225] Handling call
0204 08:52:47.961566 (+1291252us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0204 08:52:48.865799 (+904233us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":648574,"Hive Metastore.run_cpu_time_us":240,"Hive Metastore.run_wall_time_us":254829}
I20260204 08:52:48.944444 1676 catalog_manager.cc:1269] Loaded cluster ID: 41c7773c62174f70835c9cfa744e1148
I20260204 08:52:48.944592 1676 catalog_manager.cc:1562] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: loading cluster ID for follower catalog manager: success
I20260204 08:52:48.953346 1676 catalog_manager.cc:1584] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: acquiring CA information for follower catalog manager: success
I20260204 08:52:48.956009 1676 catalog_manager.cc:1612] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260204 08:52:48.992607 1722 raft_consensus.cc:493] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:48.992841 1722 raft_consensus.cc:515] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:48.993748 1722 leader_election.cc:290] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:48.994392 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb72d1554d9b4c33b6cd7c01fad765db" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:48.994405 1158 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb72d1554d9b4c33b6cd7c01fad765db" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:48.994613 1296 raft_consensus.cc:2468] T cb72d1554d9b4c33b6cd7c01fad765db P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 0.
I20260204 08:52:48.994622 1158 raft_consensus.cc:2468] T cb72d1554d9b4c33b6cd7c01fad765db P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 0.
I20260204 08:52:48.995105 958 leader_election.cc:304] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:48.995409 1722 raft_consensus.cc:2804] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:48.995530 1722 raft_consensus.cc:493] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:48.995621 1722 raft_consensus.cc:3060] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.007880 1722 raft_consensus.cc:515] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.008602 1722 leader_election.cc:290] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 election: Requested vote from peers fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:49.009212 1160 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb72d1554d9b4c33b6cd7c01fad765db" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:49.009438 1160 raft_consensus.cc:3060] T cb72d1554d9b4c33b6cd7c01fad765db P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.011533 1160 raft_consensus.cc:2468] T cb72d1554d9b4c33b6cd7c01fad765db P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 1.
I20260204 08:52:49.012156 958 leader_election.cc:304] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:49.012235 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cb72d1554d9b4c33b6cd7c01fad765db" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:49.012652 1296 raft_consensus.cc:3060] T cb72d1554d9b4c33b6cd7c01fad765db P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.014597 1722 raft_consensus.cc:2804] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:49.014775 1722 raft_consensus.cc:697] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 LEADER]: Becoming Leader. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Running, Role: LEADER
I20260204 08:52:49.015050 1722 consensus_queue.cc:237] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.107014 1296 raft_consensus.cc:2468] T cb72d1554d9b4c33b6cd7c01fad765db P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 1.
I20260204 08:52:49.109143 757 catalog_manager.cc:2731] deleted HMS catalog entry for table default.table_9e950ede78084ef18f4e1051d9a42a90 [id=c626dc1c029e4b59902b4493f4e16a78]
I20260204 08:52:49.109709 757 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:41804 (request call id 34) took 3548 ms. Trace:
I20260204 08:52:49.109817 757 rpcz_store.cc:276] 0204 08:52:45.560764 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.099403 (+1538639us) service_pool.cc:225] Handling call
0204 08:52:47.963482 (+864079us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0204 08:52:49.109659 (+1146177us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":901565,"Hive Metastore.run_cpu_time_us":250,"Hive Metastore.run_wall_time_us":243866}
I20260204 08:52:49.119916 1732 raft_consensus.cc:493] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1e719e457e94a5da71929720411ae69)
I20260204 08:52:49.120162 1732 raft_consensus.cc:515] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:49.121320 1732 leader_election.cc:290] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
I20260204 08:52:49.122031 842 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d083ecc9895345e5b06cd00ef0246c71" candidate_term: 3 candidate_status { last_received { term: 2 index: 75 } } ignore_live_leader: false dest_uuid: "c1e719e457e94a5da71929720411ae69" is_pre_election: true
I20260204 08:52:49.122030 1592 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d083ecc9895345e5b06cd00ef0246c71" candidate_term: 3 candidate_status { last_received { term: 2 index: 75 } } ignore_live_leader: false dest_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" is_pre_election: true
I20260204 08:52:49.123045 747 leader_election.cc:304] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d083ecc9895345e5b06cd00ef0246c71; no voters: c1e719e457e94a5da71929720411ae69, ebeeb5d4823b41b3a8e37546ff2f6e34
I20260204 08:52:49.128160 1732 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260204 08:52:49.166764 749 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 96ed25c622d946dfb4531ae603a80bad consensus state has no leader
I20260204 08:52:49.197091 831 catalog_manager.cc:2731] deleted HMS catalog entry for table default.table_b4244312363c4346b110598eb01428a3 [id=7d4edb505f7940668b492a9d592a3e61]
I20260204 08:52:49.198978 759 catalog_manager.cc:2174] created HMS catalog entry for table [id=e91589a913164771bb64af464d4aa1ca]
W20260204 08:52:49.200002 759 catalog_manager.cc:2202] Illegal state: an error occurred while writing to the sys-catalog: replica d083ecc9895345e5b06cd00ef0246c71 is not leader of this config: current role FOLLOWER
I20260204 08:52:49.217798 1665 raft_consensus.cc:493] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:49.218014 1665 raft_consensus.cc:515] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.219036 1665 leader_election.cc:290] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:49.220508 1020 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4c26152dcc2147d6b0cb8d1242a7a73b" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:49.220791 1020 raft_consensus.cc:2468] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:49.221478 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4c26152dcc2147d6b0cb8d1242a7a73b" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:49.221717 1296 raft_consensus.cc:2468] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:49.222541 1096 leader_election.cc:304] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:49.223588 1665 raft_consensus.cc:2804] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:49.223721 1665 raft_consensus.cc:493] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:49.223811 1665 raft_consensus.cc:3060] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.227185 1665 raft_consensus.cc:515] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.229100 1665 leader_election.cc:290] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:49.230897 1019 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4c26152dcc2147d6b0cb8d1242a7a73b" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:49.231169 1019 raft_consensus.cc:3060] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.233393 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4c26152dcc2147d6b0cb8d1242a7a73b" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:49.233647 1296 raft_consensus.cc:3060] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.233975 1019 raft_consensus.cc:2468] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:49.234692 1096 leader_election.cc:304] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:49.236159 1733 raft_consensus.cc:2804] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:49.236409 1296 raft_consensus.cc:2468] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:49.243111 1733 raft_consensus.cc:697] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Becoming Leader. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Running, Role: LEADER
I20260204 08:52:49.243619 1733 consensus_queue.cc:237] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.258253 759 catalog_manager.cc:2184] deleted orphaned HMS catalog entry for table [id=e91589a913164771bb64af464d4aa1ca]
I20260204 08:52:49.258662 759 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41804 (request call id 29) took 3747 ms. Trace:
I20260204 08:52:49.258780 759 rpcz_store.cc:276] 0204 08:52:45.510897 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:46.681534 (+1170637us) service_pool.cc:225] Handling call
0204 08:52:47.986563 (+1305029us) catalog_manager.cc:2101] Acquired catalog manager lock
0204 08:52:48.028200 (+ 41637us) catalog_manager.cc:2147] Created new table and tablet info
0204 08:52:49.198917 (+1170717us) catalog_manager.cc:2173] Created new table in HMS catalog
0204 08:52:49.199873 (+ 956us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:49.199877 (+ 4us) write_op.cc:454] Released schema lock
0204 08:52:49.200132 (+ 255us) catalog_manager.cc:2181] Rolling back HMS table creation
0204 08:52:49.258613 (+ 58481us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:49.199791 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica d083ecc9895345e5b06cd00ef0246c71 is not leader of this config: current role FOLLOWER)
0204 08:52:49.199805 (+ 14us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:49.199812 (+ 7us) write_op.cc:454] Released schema lock
0204 08:52:49.199816 (+ 4us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1080897,"Hive Metastore.run_cpu_time_us":567,"Hive Metastore.run_wall_time_us":147480,"child_traces":[["op",{}]]}
W20260204 08:52:49.259871 1353 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.104.188:38981)
I20260204 08:52:49.292358 824 catalog_manager.cc:3485] renamed table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421] in HMS: new name default.table_52f9941ea7774cceafc55a0edbdb526c
I20260204 08:52:49.327710 823 catalog_manager.cc:2174] created HMS catalog entry for table [id=f29a6ab746244116a4815a632194b3ec]
I20260204 08:52:49.360257 827 catalog_manager.cc:2174] created HMS catalog entry for table [id=12e17f7b128b4db99ae9041dba0602b5]
I20260204 08:52:49.380834 823 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:54400 (request call id 14) took 1395 ms. Trace:
I20260204 08:52:49.381040 823 rpcz_store.cc:276] 0204 08:52:47.985490 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.986086 (+ 596us) service_pool.cc:225] Handling call
0204 08:52:48.224749 (+238663us) catalog_manager.cc:2101] Acquired catalog manager lock
0204 08:52:48.225290 (+ 541us) catalog_manager.cc:2147] Created new table and tablet info
0204 08:52:49.327660 (+1102370us) catalog_manager.cc:2173] Created new table in HMS catalog
0204 08:52:49.380523 (+ 52863us) catalog_manager.cc:2207] Wrote table and tablets to system table
0204 08:52:49.380645 (+ 122us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0204 08:52:49.380782 (+ 137us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:49.332725 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0204 08:52:49.332890 (+ 165us) write_op.cc:432] Acquiring schema lock in shared mode
0204 08:52:49.332898 (+ 8us) write_op.cc:435] Acquired schema lock
0204 08:52:49.332901 (+ 3us) tablet.cc:660] Decoding operations
0204 08:52:49.333021 (+ 120us) write_op.cc:620] Acquiring the partition lock for write op
0204 08:52:49.333038 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0204 08:52:49.333044 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0204 08:52:49.333136 (+ 92us) tablet.cc:699] Row locks acquired
0204 08:52:49.333139 (+ 3us) write_op.cc:260] PREPARE: finished
0204 08:52:49.333189 (+ 50us) write_op.cc:270] Start()
0204 08:52:49.333230 (+ 41us) write_op.cc:276] Timestamp: P: 1770195169333088 usec, L: 0
0204 08:52:49.333234 (+ 4us) op_driver.cc:348] REPLICATION: starting
0204 08:52:49.333501 (+ 267us) log.cc:844] Serialized 432 byte log entry
0204 08:52:49.375173 (+ 41672us) op_driver.cc:464] REPLICATION: finished
0204 08:52:49.379724 (+ 4551us) write_op.cc:301] APPLY: starting
0204 08:52:49.379750 (+ 26us) tablet.cc:1365] starting BulkCheckPresence
0204 08:52:49.379804 (+ 54us) tablet.cc:1368] finished BulkCheckPresence
0204 08:52:49.379848 (+ 44us) tablet.cc:1370] starting ApplyRowOperation cycle
0204 08:52:49.380044 (+ 196us) tablet.cc:1381] finished ApplyRowOperation cycle
0204 08:52:49.380088 (+ 44us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0204 08:52:49.380096 (+ 8us) write_op.cc:312] APPLY: finished
0204 08:52:49.380224 (+ 128us) log.cc:844] Serialized 28 byte log entry
0204 08:52:49.380365 (+ 141us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:49.380391 (+ 26us) write_op.cc:454] Released schema lock
0204 08:52:49.380399 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"Hive Metastore.queue_time_us":1066857,"Hive Metastore.run_cpu_time_us":327,"Hive Metastore.run_wall_time_us":35289,"child_traces":[["op",{"apply.queue_time_us":4442,"num_ops":2,"prepare.queue_time_us":4067,"prepare.run_cpu_time_us":1043,"prepare.run_wall_time_us":8398,"raft.queue_time_us":120,"raft.run_cpu_time_us":909,"raft.run_wall_time_us":911,"replication_time_us":41890,"thread_start_us":3939,"threads_started":2}]]}
I20260204 08:52:49.383939 823 catalog_manager.cc:5746] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675) does not have the latest schema for tablet c9ce4ab8ce1f48868b890df81e551eed (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
W20260204 08:52:49.385015 823 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.129:59207 (request call id 71) took 789 ms (client timeout 1000 ms). Trace:
W20260204 08:52:49.385165 823 rpcz_store.cc:269] 0204 08:52:48.595236 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:49.382086 (+786850us) service_pool.cc:225] Handling call
0204 08:52:49.384936 (+ 2850us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":1688448}
W20260204 08:52:49.386792 823 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.130:39329 (request call id 65) took 791 ms (client timeout 1000 ms). Trace:
W20260204 08:52:49.386925 823 rpcz_store.cc:269] 0204 08:52:48.595388 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:49.385453 (+790065us) service_pool.cc:225] Handling call
0204 08:52:49.386731 (+ 1278us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":1}
I20260204 08:52:49.390534 823 catalog_manager.cc:5746] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491) does not have the latest schema for tablet c9ce4ab8ce1f48868b890df81e551eed (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]). Expected version 1 got 0
W20260204 08:52:49.392828 823 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.104.131:44593 (request call id 76) took 795 ms (client timeout 1000 ms). Trace:
W20260204 08:52:49.393071 823 rpcz_store.cc:269] 0204 08:52:48.597596 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:49.389567 (+791971us) service_pool.cc:225] Handling call
0204 08:52:49.392787 (+ 3220us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":1}
I20260204 08:52:49.402156 827 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:54400 (request call id 9) took 1426 ms. Trace:
I20260204 08:52:49.402369 827 rpcz_store.cc:276] 0204 08:52:47.975540 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.988019 (+ 12479us) service_pool.cc:225] Handling call
0204 08:52:48.225849 (+237830us) catalog_manager.cc:2101] Acquired catalog manager lock
0204 08:52:48.226287 (+ 438us) catalog_manager.cc:2147] Created new table and tablet info
0204 08:52:49.360183 (+1133896us) catalog_manager.cc:2173] Created new table in HMS catalog
0204 08:52:49.401877 (+ 41694us) catalog_manager.cc:2207] Wrote table and tablets to system table
0204 08:52:49.401999 (+ 122us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0204 08:52:49.402102 (+ 103us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:49.361151 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0204 08:52:49.361281 (+ 130us) write_op.cc:432] Acquiring schema lock in shared mode
0204 08:52:49.361288 (+ 7us) write_op.cc:435] Acquired schema lock
0204 08:52:49.361290 (+ 2us) tablet.cc:660] Decoding operations
0204 08:52:49.361386 (+ 96us) write_op.cc:620] Acquiring the partition lock for write op
0204 08:52:49.361398 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0204 08:52:49.361403 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0204 08:52:49.361488 (+ 85us) tablet.cc:699] Row locks acquired
0204 08:52:49.361490 (+ 2us) write_op.cc:260] PREPARE: finished
0204 08:52:49.361532 (+ 42us) write_op.cc:270] Start()
0204 08:52:49.361569 (+ 37us) write_op.cc:276] Timestamp: P: 1770195169361432 usec, L: 0
0204 08:52:49.361573 (+ 4us) op_driver.cc:348] REPLICATION: starting
0204 08:52:49.361794 (+ 221us) log.cc:844] Serialized 432 byte log entry
0204 08:52:49.401178 (+ 39384us) op_driver.cc:464] REPLICATION: finished
0204 08:52:49.401279 (+ 101us) write_op.cc:301] APPLY: starting
0204 08:52:49.401298 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
0204 08:52:49.401360 (+ 62us) tablet.cc:1368] finished BulkCheckPresence
0204 08:52:49.401396 (+ 36us) tablet.cc:1370] starting ApplyRowOperation cycle
0204 08:52:49.401526 (+ 130us) tablet.cc:1381] finished ApplyRowOperation cycle
0204 08:52:49.401564 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0204 08:52:49.401571 (+ 7us) write_op.cc:312] APPLY: finished
0204 08:52:49.401664 (+ 93us) log.cc:844] Serialized 28 byte log entry
0204 08:52:49.401753 (+ 89us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:49.401773 (+ 20us) write_op.cc:454] Released schema lock
0204 08:52:49.401788 (+ 15us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"Hive Metastore.queue_time_us":1101217,"Hive Metastore.run_cpu_time_us":278,"Hive Metastore.run_wall_time_us":32224,"child_traces":[["op",{"apply.queue_time_us":36,"apply.run_cpu_time_us":592,"apply.run_wall_time_us":593,"num_ops":2,"prepare.queue_time_us":61,"prepare.run_cpu_time_us":737,"prepare.run_wall_time_us":737,"replication_time_us":39565}]]}
I20260204 08:52:49.404538 826 catalog_manager.cc:2174] created HMS catalog entry for table [id=5a415846f1274434994325bb65b31911]
I20260204 08:52:49.408026 1002 tablet_service.cc:1505] Processing CreateTablet for tablet 5135d9b498314d9098339db68f64207d (DEFAULT_TABLE table=default.table_780ed695e5324a469614b35852a643cf [id=f29a6ab746244116a4815a632194b3ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.408041 1139 tablet_service.cc:1505] Processing CreateTablet for tablet 5135d9b498314d9098339db68f64207d (DEFAULT_TABLE table=default.table_780ed695e5324a469614b35852a643cf [id=f29a6ab746244116a4815a632194b3ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.408533 1002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5135d9b498314d9098339db68f64207d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:49.408557 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5135d9b498314d9098339db68f64207d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:49.408803 823 catalog_manager.cc:5654] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 reported cstate change: term changed from 0 to 1, leader changed from <none> to fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130). New cstate: current_term: 1 leader_uuid: "fdea56c2d1f5499087b939ec9f9dc070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: UNKNOWN } } }
I20260204 08:52:49.411293 1275 tablet_service.cc:1505] Processing CreateTablet for tablet 5135d9b498314d9098339db68f64207d (DEFAULT_TABLE table=default.table_780ed695e5324a469614b35852a643cf [id=f29a6ab746244116a4815a632194b3ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.411761 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5135d9b498314d9098339db68f64207d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:49.429013 826 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:54400 (request call id 11) took 1443 ms. Trace:
I20260204 08:52:49.429203 826 rpcz_store.cc:276] 0204 08:52:47.985173 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.990220 (+ 5047us) service_pool.cc:225] Handling call
0204 08:52:48.226777 (+236557us) catalog_manager.cc:2101] Acquired catalog manager lock
0204 08:52:48.227212 (+ 435us) catalog_manager.cc:2147] Created new table and tablet info
0204 08:52:49.404484 (+1177272us) catalog_manager.cc:2173] Created new table in HMS catalog
0204 08:52:49.428705 (+ 24221us) catalog_manager.cc:2207] Wrote table and tablets to system table
0204 08:52:49.428830 (+ 125us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0204 08:52:49.428947 (+ 117us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:49.405220 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0204 08:52:49.405335 (+ 115us) write_op.cc:432] Acquiring schema lock in shared mode
0204 08:52:49.405339 (+ 4us) write_op.cc:435] Acquired schema lock
0204 08:52:49.405341 (+ 2us) tablet.cc:660] Decoding operations
0204 08:52:49.405427 (+ 86us) write_op.cc:620] Acquiring the partition lock for write op
0204 08:52:49.405440 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0204 08:52:49.405444 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
0204 08:52:49.405557 (+ 113us) tablet.cc:699] Row locks acquired
0204 08:52:49.405559 (+ 2us) write_op.cc:260] PREPARE: finished
0204 08:52:49.405597 (+ 38us) write_op.cc:270] Start()
0204 08:52:49.405629 (+ 32us) write_op.cc:276] Timestamp: P: 1770195169405494 usec, L: 0
0204 08:52:49.405633 (+ 4us) op_driver.cc:348] REPLICATION: starting
0204 08:52:49.410651 (+ 5018us) log.cc:844] Serialized 432 byte log entry
0204 08:52:49.410969 (+ 318us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x616000041254. stack: 00007f4f8362aa86 00007f4f8362a840 00007f4f826053ad 0000000000984f94 0000000000984f5d 000000000098492a 00007f4f8f52a922 00007f4f9090d875 00007f4f9090c399 00007f4f909147c0 00007f4f909145cd 0000000000a8970c 00007f4f83670daf 00007f4f836740cb 00007f4f83673f48 0000000000a8970c
0204 08:52:49.427033 (+ 16064us) op_driver.cc:464] REPLICATION: finished
0204 08:52:49.427393 (+ 360us) write_op.cc:301] APPLY: starting
0204 08:52:49.427413 (+ 20us) tablet.cc:1365] starting BulkCheckPresence
0204 08:52:49.427460 (+ 47us) tablet.cc:1368] finished BulkCheckPresence
0204 08:52:49.427462 (+ 2us) tablet.cc:1370] starting ApplyRowOperation cycle
0204 08:52:49.427585 (+ 123us) tablet.cc:1381] finished ApplyRowOperation cycle
0204 08:52:49.427622 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0204 08:52:49.427630 (+ 8us) write_op.cc:312] APPLY: finished
0204 08:52:49.427720 (+ 90us) log.cc:844] Serialized 28 byte log entry
0204 08:52:49.427792 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:49.427815 (+ 23us) write_op.cc:454] Released schema lock
0204 08:52:49.427823 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0204 08:52:49.427899 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:49.427904 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1132589,"Hive Metastore.run_cpu_time_us":294,"Hive Metastore.run_wall_time_us":44206,"child_traces":[["op",{"apply.queue_time_us":281,"apply.run_cpu_time_us":506,"apply.run_wall_time_us":507,"num_ops":2,"prepare.queue_time_us":46,"prepare.run_cpu_time_us":1059,"prepare.run_wall_time_us":5820,"raft.queue_time_us":1509,"raft.run_cpu_time_us":375,"raft.run_wall_time_us":376,"replication_time_us":21344,"spinlock_wait_cycles":11093376}]]}
I20260204 08:52:49.460759 1745 tablet_bootstrap.cc:492] T 5135d9b498314d9098339db68f64207d P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:49.462775 1745 tablet_bootstrap.cc:654] T 5135d9b498314d9098339db68f64207d P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:49.464253 1747 tablet_bootstrap.cc:492] T 5135d9b498314d9098339db68f64207d P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:49.466785 1744 tablet_bootstrap.cc:492] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:49.466974 1747 tablet_bootstrap.cc:654] T 5135d9b498314d9098339db68f64207d P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:49.469483 1744 tablet_bootstrap.cc:654] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:49.474387 1745 tablet_bootstrap.cc:492] T 5135d9b498314d9098339db68f64207d P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:49.474673 1745 ts_tablet_manager.cc:1403] T 5135d9b498314d9098339db68f64207d P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.000s
I20260204 08:52:49.475520 1745 raft_consensus.cc:359] T 5135d9b498314d9098339db68f64207d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.475895 1745 raft_consensus.cc:385] T 5135d9b498314d9098339db68f64207d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.476047 1745 raft_consensus.cc:740] T 5135d9b498314d9098339db68f64207d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:49.476383 1745 consensus_queue.cc:260] T 5135d9b498314d9098339db68f64207d P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.477027 1745 ts_tablet_manager.cc:1434] T 5135d9b498314d9098339db68f64207d P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260204 08:52:49.484469 1747 tablet_bootstrap.cc:492] T 5135d9b498314d9098339db68f64207d P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:49.484759 1747 ts_tablet_manager.cc:1403] T 5135d9b498314d9098339db68f64207d P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.004s
I20260204 08:52:49.485790 1747 raft_consensus.cc:359] T 5135d9b498314d9098339db68f64207d P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.486097 1747 raft_consensus.cc:385] T 5135d9b498314d9098339db68f64207d P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.486243 1747 raft_consensus.cc:740] T 5135d9b498314d9098339db68f64207d P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:49.486572 1747 consensus_queue.cc:260] T 5135d9b498314d9098339db68f64207d P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.488773 1733 consensus_queue.cc:1048] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:49.493681 1744 tablet_bootstrap.cc:492] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:49.493860 1744 ts_tablet_manager.cc:1403] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.005s
I20260204 08:52:49.494743 1744 raft_consensus.cc:359] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.494931 1744 raft_consensus.cc:385] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.495033 1744 raft_consensus.cc:740] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:49.495270 1744 consensus_queue.cc:260] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.495802 1744 ts_tablet_manager.cc:1434] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260204 08:52:49.501367 1747 ts_tablet_manager.cc:1434] T 5135d9b498314d9098339db68f64207d P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.016s user 0.000s sys 0.002s
I20260204 08:52:49.503916 827 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54400:
table { table_name: "default.Table_732f1b588b344e49b2f77e17e6551853" } new_table_name: "default.Table_ae16851d63e6436692ececda3e21408e" modify_external_catalogs: true
I20260204 08:52:49.504428 826 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54400:
table { table_name: "default.Table_e7fe420574dc40cdb70c4602f5a6e0db" } new_table_name: "default.Table_2cf92b501b134cc2a2c8be92b1530908" modify_external_catalogs: true
I20260204 08:52:49.512427 1002 tablet_service.cc:1505] Processing CreateTablet for tablet 52abcf65cd4840fdb7719ada0979c7a1 (DEFAULT_TABLE table=default.table_a638f2d4f2654854afdb0b065ee83f48 [id=12e17f7b128b4db99ae9041dba0602b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.512900 1002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52abcf65cd4840fdb7719ada0979c7a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:49.517418 1139 tablet_service.cc:1505] Processing CreateTablet for tablet 52abcf65cd4840fdb7719ada0979c7a1 (DEFAULT_TABLE table=default.table_a638f2d4f2654854afdb0b065ee83f48 [id=12e17f7b128b4db99ae9041dba0602b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.517889 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52abcf65cd4840fdb7719ada0979c7a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:49.523849 823 catalog_manager.cc:5654] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129). New cstate: current_term: 1 leader_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } health_report { overall_health: HEALTHY } } }
I20260204 08:52:49.524055 1140 tablet_service.cc:1505] Processing CreateTablet for tablet 0771de581fcb4c90823dfaa8bf339cc9 (DEFAULT_TABLE table=default.table_1389d07a897b48439a706dce8abe591b [id=5a415846f1274434994325bb65b31911]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.524669 1140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0771de581fcb4c90823dfaa8bf339cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:49.529228 1000 tablet_service.cc:1505] Processing CreateTablet for tablet 0771de581fcb4c90823dfaa8bf339cc9 (DEFAULT_TABLE table=default.table_1389d07a897b48439a706dce8abe591b [id=5a415846f1274434994325bb65b31911]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.529702 1000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0771de581fcb4c90823dfaa8bf339cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:49.539536 825 catalog_manager.cc:2174] created HMS catalog entry for table [id=b3b4145c845b417aa7582e41fb48b564]
I20260204 08:52:49.542045 1276 tablet_service.cc:1505] Processing CreateTablet for tablet 0771de581fcb4c90823dfaa8bf339cc9 (DEFAULT_TABLE table=default.table_1389d07a897b48439a706dce8abe591b [id=5a415846f1274434994325bb65b31911]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.542563 1276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0771de581fcb4c90823dfaa8bf339cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:49.542045 1275 tablet_service.cc:1505] Processing CreateTablet for tablet 52abcf65cd4840fdb7719ada0979c7a1 (DEFAULT_TABLE table=default.table_a638f2d4f2654854afdb0b065ee83f48 [id=12e17f7b128b4db99ae9041dba0602b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.543574 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52abcf65cd4840fdb7719ada0979c7a1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260204 08:52:49.554600 832 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4f900fd574c849aba171e6a64bc43dbd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260204 08:52:49.556591 825 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:54400 (request call id 12) took 1571 ms. Trace:
I20260204 08:52:49.556735 825 rpcz_store.cc:276] 0204 08:52:47.985263 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:47.985323 (+ 60us) service_pool.cc:225] Handling call
0204 08:52:48.227836 (+242513us) catalog_manager.cc:2101] Acquired catalog manager lock
0204 08:52:48.228340 (+ 504us) catalog_manager.cc:2147] Created new table and tablet info
0204 08:52:49.539460 (+1311120us) catalog_manager.cc:2173] Created new table in HMS catalog
0204 08:52:49.556319 (+ 16859us) catalog_manager.cc:2207] Wrote table and tablets to system table
0204 08:52:49.556437 (+ 118us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0204 08:52:49.556545 (+ 108us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:49.541003 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0204 08:52:49.541188 (+ 185us) write_op.cc:432] Acquiring schema lock in shared mode
0204 08:52:49.541194 (+ 6us) write_op.cc:435] Acquired schema lock
0204 08:52:49.541197 (+ 3us) tablet.cc:660] Decoding operations
0204 08:52:49.547848 (+ 6651us) write_op.cc:620] Acquiring the partition lock for write op
0204 08:52:49.547865 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0204 08:52:49.547872 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0204 08:52:49.547965 (+ 93us) tablet.cc:699] Row locks acquired
0204 08:52:49.547968 (+ 3us) write_op.cc:260] PREPARE: finished
0204 08:52:49.548020 (+ 52us) write_op.cc:270] Start()
0204 08:52:49.548059 (+ 39us) write_op.cc:276] Timestamp: P: 1770195169547898 usec, L: 0
0204 08:52:49.548062 (+ 3us) op_driver.cc:348] REPLICATION: starting
0204 08:52:49.548333 (+ 271us) log.cc:844] Serialized 432 byte log entry
0204 08:52:49.555619 (+ 7286us) op_driver.cc:464] REPLICATION: finished
0204 08:52:49.555739 (+ 120us) write_op.cc:301] APPLY: starting
0204 08:52:49.555757 (+ 18us) tablet.cc:1365] starting BulkCheckPresence
0204 08:52:49.555803 (+ 46us) tablet.cc:1368] finished BulkCheckPresence
0204 08:52:49.555828 (+ 25us) tablet.cc:1370] starting ApplyRowOperation cycle
0204 08:52:49.555951 (+ 123us) tablet.cc:1381] finished ApplyRowOperation cycle
0204 08:52:49.555987 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0204 08:52:49.555994 (+ 7us) write_op.cc:312] APPLY: finished
0204 08:52:49.556102 (+ 108us) log.cc:844] Serialized 28 byte log entry
0204 08:52:49.556183 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:49.556207 (+ 24us) write_op.cc:454] Released schema lock
0204 08:52:49.556215 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"Hive Metastore.queue_time_us":1175689,"Hive Metastore.run_cpu_time_us":309,"Hive Metastore.run_wall_time_us":135174,"child_traces":[["op",{"apply.queue_time_us":68,"num_ops":2,"prepare.queue_time_us":186,"prepare.run_cpu_time_us":916,"prepare.run_wall_time_us":7441,"replication_time_us":7528}]]}
F20260204 08:52:49.560977 888 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 23, previous 23; dumping first 4 out of 14 received events: NotificationEvent(eventId=21, eventTime=1770195168, eventType=DROP_TABLE, dbName=default, tableName=table_85f07b4907b24f1ca63791592455b59c, message={"server":"","servicePrincipal":"","db":"default","table":"table_85f07b4907b24f1ca63791592455b59c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_85f07b4907b24f1ca63791592455b59c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195167},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_85f07b4907b24f1ca63791592455b59c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_85f07b4907b24f1ca63791592455b59c\",\"kudu.table_id\":\"158d366d5ccf4b78b73903776a30cb35\",\"transient_lastDdlTime\":\"1770195167\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=22, eventTime=1770195168, eventType=CREATE_TABLE, dbName=default, tableName=table_97ec63740f2b4928af12e35b5eaa29cd, message={"server":"","servicePrincipal":"","db":"default","table":"table_97ec63740f2b4928af12e35b5eaa29cd","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_97ec63740f2b4928af12e35b5eaa29cd\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195168},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_97ec63740f2b4928af12e35b5eaa29cd\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_97ec63740f2b4928af12e35b5eaa29cd\",\"kudu.table_id\":\"2a58ed39876b46aba637d263234e55d9\",\"transient_lastDdlTime\":\"1770195168\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=23, eventTime=1770195168, eventType=DROP_TABLE, dbName=default, tableName=table_279632cf491c4198ad9b91a114c3fc33, message={"server":"","servicePrincipal":"","db":"default","table":"table_279632cf491c4198ad9b91a114c3fc33","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_279632cf491c4198ad9b91a114c3fc33\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195165},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_279632cf491c4198ad9b91a114c3fc33\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_279632cf491c4198ad9b91a114c3fc33\",\"kudu.table_id\":\"da8943eba12e426ba3bac81a64141294\",\"transient_lastDdlTime\":\"1770195165\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=23, eventTime=1770195168, eventType=CREATE_TABLE, dbName=default, tableName=table_477defa30c2c40e99360b505ef988a80, message={"server":"","servicePrincipal":"","db":"default","table":"table_477defa30c2c40e99360b505ef988a80","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_477defa30c2c40e99360b505ef988a80\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195168},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_477defa30c2c40e99360b505ef988a80\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_477defa30c2c40e99360b505ef988a80\",\"kudu.table_id\":\"b1e79546138b4dcb9c5a68977af78bf0\",\"transient_lastDdlTime\":\"1770195168\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f4f808dadcd google::LogMessage::Fail() at ??:0
I20260204 08:52:49.573387 1744 tablet_bootstrap.cc:492] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
@ 0x7f4f808deb93 google::LogMessage::SendToLog() at ??:0
I20260204 08:52:49.575212 1744 tablet_bootstrap.cc:654] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
@ 0x7f4f808da7cc google::LogMessage::Flush() at ??:0
I20260204 08:52:49.576416 1001 tablet_service.cc:1505] Processing CreateTablet for tablet 822cc58f9ac042bf8b21c8ac3fc30ec2 (DEFAULT_TABLE table=default.table_f087faf331654fae99b16adcfb5f705f [id=b3b4145c845b417aa7582e41fb48b564]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.576829 1001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 822cc58f9ac042bf8b21c8ac3fc30ec2. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f4f808daa11 google::LogMessage::~LogMessage() at ??:0
I20260204 08:52:49.587934 1747 tablet_bootstrap.cc:492] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:49.587957 1733 consensus_queue.cc:1048] T d3c5509788ad4f8c858db788681b0920 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:49.588452 1745 tablet_bootstrap.cc:492] T 52abcf65cd4840fdb7719ada0979c7a1 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:49.589941 1747 tablet_bootstrap.cc:654] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:49.595137 825 master_service.cc:954] ReplaceTablet: received request to replace tablet bf91146e638f43e0b493a5732f6174be from {username='slave'} at 127.0.0.1:54400
I20260204 08:52:49.595575 1275 tablet_service.cc:1505] Processing CreateTablet for tablet 822cc58f9ac042bf8b21c8ac3fc30ec2 (DEFAULT_TABLE table=default.table_f087faf331654fae99b16adcfb5f705f [id=b3b4145c845b417aa7582e41fb48b564]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.596010 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 822cc58f9ac042bf8b21c8ac3fc30ec2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:49.596680 823 master_service.cc:954] ReplaceTablet: received request to replace tablet 7121dc10d79548169242f5b65331c08a from {username='slave'} at 127.0.0.1:54400
I20260204 08:52:49.598734 1747 tablet_bootstrap.cc:492] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:49.598928 1747 ts_tablet_manager.cc:1403] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260204 08:52:49.599783 1747 raft_consensus.cc:359] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.599993 1747 raft_consensus.cc:385] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.600077 1747 raft_consensus.cc:740] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:49.600425 1747 consensus_queue.cc:260] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.601466 1745 tablet_bootstrap.cc:654] T 52abcf65cd4840fdb7719ada0979c7a1 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:49.602735 1744 tablet_bootstrap.cc:492] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:49.602905 1747 ts_tablet_manager.cc:1434] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260204 08:52:49.602922 1744 ts_tablet_manager.cc:1403] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.030s user 0.009s sys 0.002s
I20260204 08:52:49.603247 1747 tablet_bootstrap.cc:492] T 52abcf65cd4840fdb7719ada0979c7a1 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:49.603824 1744 raft_consensus.cc:359] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.604045 1744 raft_consensus.cc:385] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.605000 1747 tablet_bootstrap.cc:654] T 52abcf65cd4840fdb7719ada0979c7a1 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:49.605135 1744 raft_consensus.cc:740] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:49.605443 1744 consensus_queue.cc:260] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.605950 1744 ts_tablet_manager.cc:1434] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260204 08:52:49.606325 1744 tablet_bootstrap.cc:492] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
@ 0x7f4f9730e7f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260204 08:52:49.604929 1140 tablet_service.cc:1505] Processing CreateTablet for tablet 822cc58f9ac042bf8b21c8ac3fc30ec2 (DEFAULT_TABLE table=default.table_f087faf331654fae99b16adcfb5f705f [id=b3b4145c845b417aa7582e41fb48b564]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.608781 1140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 822cc58f9ac042bf8b21c8ac3fc30ec2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:49.613121 1744 tablet_bootstrap.cc:654] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
@ 0x7f4f9730d0a1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f4f97314d1c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260204 08:52:49.628880 1745 tablet_bootstrap.cc:492] T 52abcf65cd4840fdb7719ada0979c7a1 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:49.629076 1745 ts_tablet_manager.cc:1403] T 52abcf65cd4840fdb7719ada0979c7a1 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.041s user 0.006s sys 0.000s
@ 0x7f4f97314b99 std::_Function_handler<>::_M_invoke() at ??:0
I20260204 08:52:49.632189 1745 raft_consensus.cc:359] T 52abcf65cd4840fdb7719ada0979c7a1 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.632474 1745 raft_consensus.cc:385] T 52abcf65cd4840fdb7719ada0979c7a1 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.632557 1745 raft_consensus.cc:740] T 52abcf65cd4840fdb7719ada0979c7a1 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
@ 0xa8970d std::function<>::operator()() at ??:?
I20260204 08:52:49.635171 825 catalog_manager.cc:5941] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: Sending DeleteTablet for 3 replicas of tablet bf91146e638f43e0b493a5732f6174be
I20260204 08:52:49.637035 823 catalog_manager.cc:5941] T 00000000000000000000000000000000 P c1e719e457e94a5da71929720411ae69: Sending DeleteTablet for 3 replicas of tablet 7121dc10d79548169242f5b65331c08a
I20260204 08:52:49.637539 823 catalog_manager.cc:6724] ReplaceTablet: tablet 7121dc10d79548169242f5b65331c08a deleted and replaced by tablet 350b048c73bd4208a7aa70b3521869e1
I20260204 08:52:49.637730 823 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:54400 (request call id 26) took 1345 ms. Trace:
I20260204 08:52:49.637820 823 rpcz_store.cc:276] 0204 08:52:48.292319 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:49.596570 (+1304251us) service_pool.cc:225] Handling call
0204 08:52:49.635010 (+ 38440us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:49.635015 (+ 5us) write_op.cc:454] Released schema lock
0204 08:52:49.637633 (+ 2618us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:49.597982 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0204 08:52:49.598117 (+ 135us) write_op.cc:432] Acquiring schema lock in shared mode
0204 08:52:49.598124 (+ 7us) write_op.cc:435] Acquired schema lock
0204 08:52:49.598126 (+ 2us) tablet.cc:660] Decoding operations
0204 08:52:49.598241 (+ 115us) write_op.cc:620] Acquiring the partition lock for write op
0204 08:52:49.598255 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0204 08:52:49.598260 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0204 08:52:49.598371 (+ 111us) tablet.cc:699] Row locks acquired
0204 08:52:49.598374 (+ 3us) write_op.cc:260] PREPARE: finished
0204 08:52:49.598417 (+ 43us) write_op.cc:270] Start()
0204 08:52:49.598455 (+ 38us) write_op.cc:276] Timestamp: P: 1770195169598292 usec, L: 0
0204 08:52:49.598458 (+ 3us) op_driver.cc:348] REPLICATION: starting
0204 08:52:49.598676 (+ 218us) log.cc:844] Serialized 661 byte log entry
0204 08:52:49.606560 (+ 7884us) op_driver.cc:464] REPLICATION: finished
0204 08:52:49.613654 (+ 7094us) write_op.cc:301] APPLY: starting
0204 08:52:49.613677 (+ 23us) tablet.cc:1365] starting BulkCheckPresence
0204 08:52:49.613737 (+ 60us) tablet.cc:1368] finished BulkCheckPresence
0204 08:52:49.613773 (+ 36us) tablet.cc:1370] starting ApplyRowOperation cycle
0204 08:52:49.613928 (+ 155us) tablet.cc:1381] finished ApplyRowOperation cycle
0204 08:52:49.614005 (+ 77us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0204 08:52:49.614013 (+ 8us) write_op.cc:312] APPLY: finished
0204 08:52:49.614114 (+ 101us) log.cc:844] Serialized 28 byte log entry
0204 08:52:49.614209 (+ 95us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:49.614234 (+ 25us) write_op.cc:454] Released schema lock
0204 08:52:49.614242 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":6999,"apply.run_cpu_time_us":672,"apply.run_wall_time_us":673,"num_ops":2,"prepare.queue_time_us":196,"prepare.run_cpu_time_us":832,"prepare.run_wall_time_us":832,"raft.queue_time_us":635,"raft.run_cpu_time_us":674,"raft.run_wall_time_us":673,"replication_time_us":8067,"spinlock_wait_cycles":6528}]]}
I20260204 08:52:49.639734 825 catalog_manager.cc:6724] ReplaceTablet: tablet bf91146e638f43e0b493a5732f6174be deleted and replaced by tablet e31d044c9bb14ceb8d637977966630b9
I20260204 08:52:49.639884 825 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:54400 (request call id 25) took 1347 ms. Trace:
I20260204 08:52:49.640342 1138 tablet_service.cc:1552] Processing DeleteTablet for tablet 7121dc10d79548169242f5b65331c08a with delete_type TABLET_DATA_DELETED (replaced by tablet 350b048c73bd4208a7aa70b3521869e1) from {username='slave'} at 127.0.0.1:57288
I20260204 08:52:49.640342 1139 tablet_service.cc:1552] Processing DeleteTablet for tablet bf91146e638f43e0b493a5732f6174be with delete_type TABLET_DATA_DELETED (replaced by tablet e31d044c9bb14ceb8d637977966630b9) from {username='slave'} at 127.0.0.1:57288
I20260204 08:52:49.639967 825 rpcz_store.cc:276] 0204 08:52:48.292148 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:49.595067 (+1302919us) service_pool.cc:225] Handling call
0204 08:52:49.634668 (+ 39601us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:49.634676 (+ 8us) write_op.cc:454] Released schema lock
0204 08:52:49.639832 (+ 5156us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:49.620646 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0204 08:52:49.620910 (+ 264us) write_op.cc:432] Acquiring schema lock in shared mode
0204 08:52:49.620917 (+ 7us) write_op.cc:435] Acquired schema lock
0204 08:52:49.620919 (+ 2us) tablet.cc:660] Decoding operations
0204 08:52:49.621099 (+ 180us) write_op.cc:620] Acquiring the partition lock for write op
0204 08:52:49.621115 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0204 08:52:49.621121 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0204 08:52:49.621212 (+ 91us) tablet.cc:699] Row locks acquired
0204 08:52:49.621215 (+ 3us) write_op.cc:260] PREPARE: finished
0204 08:52:49.621264 (+ 49us) write_op.cc:270] Start()
0204 08:52:49.621305 (+ 41us) write_op.cc:276] Timestamp: P: 1770195169621136 usec, L: 0
0204 08:52:49.621308 (+ 3us) op_driver.cc:348] REPLICATION: starting
0204 08:52:49.621542 (+ 234us) log.cc:844] Serialized 661 byte log entry
0204 08:52:49.628103 (+ 6561us) op_driver.cc:464] REPLICATION: finished
0204 08:52:49.630555 (+ 2452us) write_op.cc:301] APPLY: starting
0204 08:52:49.630575 (+ 20us) tablet.cc:1365] starting BulkCheckPresence
0204 08:52:49.630620 (+ 45us) tablet.cc:1368] finished BulkCheckPresence
0204 08:52:49.630660 (+ 40us) tablet.cc:1370] starting ApplyRowOperation cycle
0204 08:52:49.630773 (+ 113us) tablet.cc:1381] finished ApplyRowOperation cycle
0204 08:52:49.630836 (+ 63us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0204 08:52:49.630844 (+ 8us) write_op.cc:312] APPLY: finished
0204 08:52:49.630939 (+ 95us) log.cc:844] Serialized 28 byte log entry
0204 08:52:49.631045 (+ 106us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:49.631069 (+ 24us) write_op.cc:454] Released schema lock
0204 08:52:49.631078 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2379,"apply.run_cpu_time_us":600,"apply.run_wall_time_us":599,"mutex_wait_us":22595,"num_ops":2,"prepare.queue_time_us":22689,"prepare.run_cpu_time_us":963,"prepare.run_wall_time_us":1046,"raft.queue_time_us":536,"raft.run_cpu_time_us":555,"raft.run_wall_time_us":553,"replication_time_us":6760}]]}
I20260204 08:52:49.641064 1000 tablet_service.cc:1552] Processing DeleteTablet for tablet bf91146e638f43e0b493a5732f6174be with delete_type TABLET_DATA_DELETED (replaced by tablet e31d044c9bb14ceb8d637977966630b9) from {username='slave'} at 127.0.0.1:33412
I20260204 08:52:49.641541 1002 tablet_service.cc:1552] Processing DeleteTablet for tablet 7121dc10d79548169242f5b65331c08a with delete_type TABLET_DATA_DELETED (replaced by tablet 350b048c73bd4208a7aa70b3521869e1) from {username='slave'} at 127.0.0.1:33412
I20260204 08:52:49.641562 1276 tablet_service.cc:1552] Processing DeleteTablet for tablet bf91146e638f43e0b493a5732f6174be with delete_type TABLET_DATA_DELETED (replaced by tablet e31d044c9bb14ceb8d637977966630b9) from {username='slave'} at 127.0.0.1:36752
I20260204 08:52:49.642012 1274 tablet_service.cc:1552] Processing DeleteTablet for tablet 7121dc10d79548169242f5b65331c08a with delete_type TABLET_DATA_DELETED (replaced by tablet 350b048c73bd4208a7aa70b3521869e1) from {username='slave'} at 127.0.0.1:36752
I20260204 08:52:49.646544 825 master_service.cc:954] ReplaceTablet: received request to replace tablet bf91146e638f43e0b493a5732f6174be from {username='slave'} at 127.0.0.1:54400
I20260204 08:52:49.646948 825 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:54400 (request call id 27) took 1354 ms. Trace:
I20260204 08:52:49.647059 825 rpcz_store.cc:276] 0204 08:52:48.292418 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:49.646438 (+1354020us) service_pool.cc:225] Handling call
0204 08:52:49.646911 (+ 473us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260204 08:52:49.647460 825 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54400:
name: "default.Table_c2dfe1ff212d4d2b8d46f11cb24ac67e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:49.656774 1767 tablet_replica.cc:333] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:52:49.657047 1767 raft_consensus.cc:2243] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:49.657274 1767 raft_consensus.cc:2272] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:49.661525 1139 tablet_service.cc:1505] Processing CreateTablet for tablet 350b048c73bd4208a7aa70b3521869e1 (DEFAULT_TABLE table=default.table_f8273ef1a407421085918a8e3827f5f6 [id=2df47e721cd642e4b6fd5e61e172749c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.662111 1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 350b048c73bd4208a7aa70b3521869e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:49.662806 1765 tablet_replica.cc:333] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:52:49.663025 1765 raft_consensus.cc:2243] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:49.663278 1765 raft_consensus.cc:2272] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:49.663874 1765 ts_tablet_manager.cc:1916] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:49.666422 1138 tablet_service.cc:1505] Processing CreateTablet for tablet e31d044c9bb14ceb8d637977966630b9 (DEFAULT_TABLE table=default.table_c97013dd16f84d8ba7cd528295f5ad49 [id=5f9f5b0850774ff9b8ecb93e141ff768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.666831 1138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e31d044c9bb14ceb8d637977966630b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:49.668416 832 catalog_manager.cc:2174] created HMS catalog entry for table [id=9427e77d6a3842a7a2665d99c8ebb99e]
I20260204 08:52:49.670991 1276 tablet_service.cc:1505] Processing CreateTablet for tablet e31d044c9bb14ceb8d637977966630b9 (DEFAULT_TABLE table=default.table_c97013dd16f84d8ba7cd528295f5ad49 [id=5f9f5b0850774ff9b8ecb93e141ff768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.671483 1276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e31d044c9bb14ceb8d637977966630b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:49.677269 832 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:54400 (request call id 16) took 1631 ms. Trace:
I20260204 08:52:49.677322 1606 raft_consensus.cc:493] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:49.677474 1606 raft_consensus.cc:515] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.677402 832 rpcz_store.cc:276] 0204 08:52:48.045538 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:48.093099 (+ 47561us) service_pool.cc:225] Handling call
0204 08:52:48.176720 (+ 83621us) spinlock_profiling.cc:243] Waited 83.3 ms on lock 0x616000041254. stack: 00007f4f8362aa86 00007f4f8362a840 00007f4f826053ad 0000000000984f94 0000000000984f5d 00007f4f929916bd 00007f4f929917b3 00007f4f8f558521 00007f4f972236fd 00007f4f973afd93 00007f4f8deae7c6 00007f4f8deae4f5 00007f4f8a62a55e 00007f4f8a62943e 00007f4f8a62f675 00007f4f8a62ff9f
0204 08:52:49.554732 (+1378012us) catalog_manager.cc:2101] Acquired catalog manager lock
0204 08:52:49.555238 (+ 506us) catalog_manager.cc:2147] Created new table and tablet info
0204 08:52:49.668377 (+113139us) catalog_manager.cc:2173] Created new table in HMS catalog
0204 08:52:49.676982 (+ 8605us) catalog_manager.cc:2207] Wrote table and tablets to system table
0204 08:52:49.677094 (+ 112us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0204 08:52:49.677198 (+ 104us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0204 08:52:49.669084 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0204 08:52:49.669201 (+ 117us) write_op.cc:432] Acquiring schema lock in shared mode
0204 08:52:49.669208 (+ 7us) write_op.cc:435] Acquired schema lock
0204 08:52:49.669211 (+ 3us) tablet.cc:660] Decoding operations
0204 08:52:49.669344 (+ 133us) write_op.cc:620] Acquiring the partition lock for write op
0204 08:52:49.669358 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0204 08:52:49.669363 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0204 08:52:49.669441 (+ 78us) tablet.cc:699] Row locks acquired
0204 08:52:49.669443 (+ 2us) write_op.cc:260] PREPARE: finished
0204 08:52:49.669481 (+ 38us) write_op.cc:270] Start()
0204 08:52:49.669515 (+ 34us) write_op.cc:276] Timestamp: P: 1770195169669352 usec, L: 0
0204 08:52:49.669519 (+ 4us) op_driver.cc:348] REPLICATION: starting
0204 08:52:49.669704 (+ 185us) log.cc:844] Serialized 432 byte log entry
0204 08:52:49.673168 (+ 3464us) op_driver.cc:464] REPLICATION: finished
0204 08:52:49.676401 (+ 3233us) write_op.cc:301] APPLY: starting
0204 08:52:49.676419 (+ 18us) tablet.cc:1365] starting BulkCheckPresence
0204 08:52:49.676460 (+ 41us) tablet.cc:1368] finished BulkCheckPresence
0204 08:52:49.676497 (+ 37us) tablet.cc:1370] starting ApplyRowOperation cycle
0204 08:52:49.676597 (+ 100us) tablet.cc:1381] finished ApplyRowOperation cycle
0204 08:52:49.676629 (+ 32us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0204 08:52:49.676637 (+ 8us) write_op.cc:312] APPLY: finished
0204 08:52:49.676717 (+ 80us) log.cc:844] Serialized 28 byte log entry
0204 08:52:49.676785 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:49.676806 (+ 21us) write_op.cc:454] Released schema lock
0204 08:52:49.676812 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0204 08:52:49.676882 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0204 08:52:49.676886 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":4890,"Hive Metastore.run_cpu_time_us":272,"Hive Metastore.run_wall_time_us":107957,"spinlock_wait_cycles":191490944,"child_traces":[["op",{"apply.queue_time_us":3161,"apply.run_cpu_time_us":480,"apply.run_wall_time_us":481,"num_ops":2,"prepare.queue_time_us":59,"prepare.run_cpu_time_us":741,"prepare.run_wall_time_us":741,"raft.queue_time_us":420,"raft.run_cpu_time_us":499,"raft.run_wall_time_us":498,"replication_time_us":3618}]]}
I20260204 08:52:49.678038 1747 tablet_bootstrap.cc:492] T 52abcf65cd4840fdb7719ada0979c7a1 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:49.678200 1747 ts_tablet_manager.cc:1403] T 52abcf65cd4840fdb7719ada0979c7a1 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.075s user 0.011s sys 0.010s
I20260204 08:52:49.678982 1747 raft_consensus.cc:359] T 52abcf65cd4840fdb7719ada0979c7a1 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.679255 1747 raft_consensus.cc:385] T 52abcf65cd4840fdb7719ada0979c7a1 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.679340 1747 raft_consensus.cc:740] T 52abcf65cd4840fdb7719ada0979c7a1 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:49.679598 1747 consensus_queue.cc:260] T 52abcf65cd4840fdb7719ada0979c7a1 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.679802 823 rpcz_store.cc:275] Call kudu.master.MasterService.ConnectToMaster from 127.0.0.1:54400 (request call id 29) took 1326 ms. Trace:
I20260204 08:52:49.679913 823 rpcz_store.cc:276] 0204 08:52:48.353786 (+ 0us) service_pool.cc:168] Inserting onto call queue
0204 08:52:49.679043 (+1325257us) service_pool.cc:225] Handling call
0204 08:52:49.679729 (+ 686us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260204 08:52:49.680207 823 master_service.cc:954] ReplaceTablet: received request to replace tablet d3c5509788ad4f8c858db788681b0920 from {username='slave'} at 127.0.0.1:54400
I20260204 08:52:49.681931 1606 leader_election.cc:290] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:49.682242 1002 tablet_service.cc:1505] Processing CreateTablet for tablet e31d044c9bb14ceb8d637977966630b9 (DEFAULT_TABLE table=default.table_c97013dd16f84d8ba7cd528295f5ad49 [id=5f9f5b0850774ff9b8ecb93e141ff768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.682734 1002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e31d044c9bb14ceb8d637977966630b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:49.683807 1747 ts_tablet_manager.cc:1434] T 52abcf65cd4840fdb7719ada0979c7a1 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
@ 0x7f4f8364ff99 kudu::Thread::SuperviseThread() at ??:0
I20260204 08:52:49.686932 1747 tablet_bootstrap.cc:492] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
@ 0x7f4f86c5c6db start_thread at ??:0
@ 0x7f4f7f78b71f clone at ??:0
I20260204 08:52:49.688767 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5135d9b498314d9098339db68f64207d" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:49.688967 1747 tablet_bootstrap.cc:654] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:49.689013 1296 raft_consensus.cc:2468] T 5135d9b498314d9098339db68f64207d P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 0.
I20260204 08:52:49.689911 958 leader_election.cc:304] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:49.690829 1275 tablet_service.cc:1505] Processing CreateTablet for tablet 350b048c73bd4208a7aa70b3521869e1 (DEFAULT_TABLE table=default.table_f8273ef1a407421085918a8e3827f5f6 [id=2df47e721cd642e4b6fd5e61e172749c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.690834 1000 tablet_service.cc:1505] Processing CreateTablet for tablet 350b048c73bd4208a7aa70b3521869e1 (DEFAULT_TABLE table=default.table_f8273ef1a407421085918a8e3827f5f6 [id=2df47e721cd642e4b6fd5e61e172749c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:49.691186 1000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 350b048c73bd4208a7aa70b3521869e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:49.691237 1275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 350b048c73bd4208a7aa70b3521869e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:49.691700 1606 raft_consensus.cc:2804] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:49.692164 1606 raft_consensus.cc:493] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:49.692417 1538 raft_consensus.cc:493] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:49.692286 1606 raft_consensus.cc:3060] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.692574 1538 raft_consensus.cc:515] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.692533 1745 consensus_queue.cc:260] T 52abcf65cd4840fdb7719ada0979c7a1 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.693153 1745 ts_tablet_manager.cc:1434] T 52abcf65cd4840fdb7719ada0979c7a1 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.064s user 0.002s sys 0.000s
I20260204 08:52:49.693216 1538 leader_election.cc:290] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:49.695742 1160 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0771de581fcb4c90823dfaa8bf339cc9" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:49.696192 1020 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0771de581fcb4c90823dfaa8bf339cc9" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
W20260204 08:52:49.696478 1232 leader_election.cc:343] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737): Illegal state: must be running to vote when last-logged opid is not known
I20260204 08:52:49.696570 1020 raft_consensus.cc:2468] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:49.698182 1606 raft_consensus.cc:515] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.698832 1606 leader_election.cc:290] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 election: Requested vote from peers fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:49.699254 1232 leader_election.cc:304] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters: fdea56c2d1f5499087b939ec9f9dc070
I20260204 08:52:49.699436 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5135d9b498314d9098339db68f64207d" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:49.699673 1296 raft_consensus.cc:3060] T 5135d9b498314d9098339db68f64207d P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.700635 1538 raft_consensus.cc:2804] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:49.700744 1538 raft_consensus.cc:493] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:49.700853 1538 raft_consensus.cc:3060] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.702876 1296 raft_consensus.cc:2468] T 5135d9b498314d9098339db68f64207d P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 1.
I20260204 08:52:49.706481 958 leader_election.cc:304] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:49.706949 1606 raft_consensus.cc:2804] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:49.707754 1606 raft_consensus.cc:697] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 LEADER]: Becoming Leader. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Running, Role: LEADER
I20260204 08:52:49.708643 1538 raft_consensus.cc:515] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.708051 1606 consensus_queue.cc:237] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.709353 1538 leader_election.cc:290] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:49.713564 1766 tablet_replica.cc:333] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:52:49.713789 1766 raft_consensus.cc:2243] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:49.714033 1766 raft_consensus.cc:2272] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:49.714686 1766 ts_tablet_manager.cc:1916] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:49.717572 1767 ts_tablet_manager.cc:1916] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:49.717566 1158 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5135d9b498314d9098339db68f64207d" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:49.717847 1158 raft_consensus.cc:2468] T 5135d9b498314d9098339db68f64207d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 0.
I20260204 08:52:49.718201 1160 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5135d9b498314d9098339db68f64207d" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:49.718379 1160 raft_consensus.cc:3060] T 5135d9b498314d9098339db68f64207d P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.726905 1744 tablet_bootstrap.cc:492] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:49.727279 1744 ts_tablet_manager.cc:1403] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.121s user 0.005s sys 0.012s
I20260204 08:52:49.728207 1744 raft_consensus.cc:359] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.728478 1744 raft_consensus.cc:385] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.728569 1744 raft_consensus.cc:740] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:49.728816 1744 consensus_queue.cc:260] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.731462 1160 raft_consensus.cc:2468] T 5135d9b498314d9098339db68f64207d P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 1.
I20260204 08:52:49.731604 1744 ts_tablet_manager.cc:1434] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260204 08:52:49.731933 1744 tablet_bootstrap.cc:492] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:49.733816 1744 tablet_bootstrap.cc:654] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:49.739197 1745 tablet_bootstrap.cc:492] T 0771de581fcb4c90823dfaa8bf339cc9 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:49.743230 1020 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0771de581fcb4c90823dfaa8bf339cc9" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:49.743475 1020 raft_consensus.cc:3060] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.746358 1020 raft_consensus.cc:2468] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:49.747406 1767 ts_tablet_manager.cc:1929] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:49.747556 1767 log.cc:1199] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/bf91146e638f43e0b493a5732f6174be
I20260204 08:52:49.747661 1160 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0771de581fcb4c90823dfaa8bf339cc9" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:49.747732 1232 leader_election.cc:304] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:49.748026 1767 ts_tablet_manager.cc:1950] T bf91146e638f43e0b493a5732f6174be P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:52:49.748087 1538 raft_consensus.cc:2804] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:49.748322 1538 raft_consensus.cc:697] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Becoming Leader. State: Replica: 1234acd7d0db405085a5d95795007134, State: Running, Role: LEADER
W20260204 08:52:49.748350 1232 leader_election.cc:343] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737): Illegal state: must be running to vote when last-logged opid is not known
I20260204 08:52:49.748835 1538 consensus_queue.cc:237] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.749488 1747 tablet_bootstrap.cc:492] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:49.749651 1747 ts_tablet_manager.cc:1403] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.063s user 0.003s sys 0.000s
I20260204 08:52:49.750524 1747 raft_consensus.cc:359] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.750738 1747 raft_consensus.cc:385] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.750809 1747 raft_consensus.cc:740] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:49.751811 1766 ts_tablet_manager.cc:1929] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:49.751906 1766 log.cc:1199] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/bf91146e638f43e0b493a5732f6174be
I20260204 08:52:49.752288 1766 ts_tablet_manager.cc:1950] T bf91146e638f43e0b493a5732f6174be P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:52:49.755566 1745 tablet_bootstrap.cc:654] T 0771de581fcb4c90823dfaa8bf339cc9 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:49.756366 1747 consensus_queue.cc:260] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.757009 1766 tablet_replica.cc:333] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:52:49.757018 1747 ts_tablet_manager.cc:1434] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260204 08:52:49.757423 1747 tablet_bootstrap.cc:492] T e31d044c9bb14ceb8d637977966630b9 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:49.757691 1744 tablet_bootstrap.cc:492] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:49.757867 1744 ts_tablet_manager.cc:1403] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.000s
I20260204 08:52:49.758628 1744 raft_consensus.cc:359] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.758810 1744 raft_consensus.cc:385] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.758868 1744 raft_consensus.cc:740] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:49.759207 1747 tablet_bootstrap.cc:654] T e31d044c9bb14ceb8d637977966630b9 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:49.759119 1744 consensus_queue.cc:260] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.763089 1744 ts_tablet_manager.cc:1434] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260204 08:52:49.764190 1765 ts_tablet_manager.cc:1929] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260204 08:52:49.766690 1744 tablet_bootstrap.cc:492] T e31d044c9bb14ceb8d637977966630b9 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:49.768477 1744 tablet_bootstrap.cc:654] T e31d044c9bb14ceb8d637977966630b9 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:49.770125 1767 tablet_replica.cc:333] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:52:49.770364 1767 raft_consensus.cc:2243] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:52:49.770401 1745 tablet_bootstrap.cc:492] T 0771de581fcb4c90823dfaa8bf339cc9 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:49.770563 1745 ts_tablet_manager.cc:1403] T 0771de581fcb4c90823dfaa8bf339cc9 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.077s user 0.004s sys 0.000s
I20260204 08:52:49.771085 1766 raft_consensus.cc:2243] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:49.771309 1766 raft_consensus.cc:2272] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:49.771360 1745 raft_consensus.cc:359] T 0771de581fcb4c90823dfaa8bf339cc9 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.771540 1745 raft_consensus.cc:385] T 0771de581fcb4c90823dfaa8bf339cc9 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.771620 1745 raft_consensus.cc:740] T 0771de581fcb4c90823dfaa8bf339cc9 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:49.771821 1765 log.cc:1199] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/7121dc10d79548169242f5b65331c08a
I20260204 08:52:49.771871 1766 ts_tablet_manager.cc:1916] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:49.772322 1765 ts_tablet_manager.cc:1950] T 7121dc10d79548169242f5b65331c08a P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:52:49.773263 1765 tablet_replica.cc:333] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:52:49.773474 1765 raft_consensus.cc:2243] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:52:49.773769 1765 raft_consensus.cc:2272] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:49.774322 1765 ts_tablet_manager.cc:1916] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:49.776427 1767 raft_consensus.cc:2272] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:49.777050 1767 ts_tablet_manager.cc:1916] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:49.778951 1744 tablet_bootstrap.cc:492] T e31d044c9bb14ceb8d637977966630b9 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:49.779103 1744 ts_tablet_manager.cc:1403] T e31d044c9bb14ceb8d637977966630b9 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.002s
I20260204 08:52:49.779866 1744 raft_consensus.cc:359] T e31d044c9bb14ceb8d637977966630b9 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.780071 1744 raft_consensus.cc:385] T e31d044c9bb14ceb8d637977966630b9 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.780144 1744 raft_consensus.cc:740] T e31d044c9bb14ceb8d637977966630b9 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:49.780731 1765 ts_tablet_manager.cc:1929] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:49.780865 1765 log.cc:1199] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/bf91146e638f43e0b493a5732f6174be
I20260204 08:52:49.781293 1765 ts_tablet_manager.cc:1950] T bf91146e638f43e0b493a5732f6174be P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:52:49.771839 1745 consensus_queue.cc:260] T 0771de581fcb4c90823dfaa8bf339cc9 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.782398 1747 tablet_bootstrap.cc:492] T e31d044c9bb14ceb8d637977966630b9 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:49.782558 1747 ts_tablet_manager.cc:1403] T e31d044c9bb14ceb8d637977966630b9 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.002s
I20260204 08:52:49.782868 1745 ts_tablet_manager.cc:1434] T 0771de581fcb4c90823dfaa8bf339cc9 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260204 08:52:49.783329 1745 tablet_bootstrap.cc:492] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:49.783447 1747 raft_consensus.cc:359] T e31d044c9bb14ceb8d637977966630b9 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.789286 1745 tablet_bootstrap.cc:654] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:49.790300 1767 ts_tablet_manager.cc:1929] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260204 08:52:49.794566 1747 raft_consensus.cc:385] T e31d044c9bb14ceb8d637977966630b9 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.794723 1747 raft_consensus.cc:740] T e31d044c9bb14ceb8d637977966630b9 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:49.794756 1767 log.cc:1199] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/7121dc10d79548169242f5b65331c08a
I20260204 08:52:49.795073 1747 consensus_queue.cc:260] T e31d044c9bb14ceb8d637977966630b9 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.795296 1767 ts_tablet_manager.cc:1950] T 7121dc10d79548169242f5b65331c08a P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:52:49.795661 1747 ts_tablet_manager.cc:1434] T e31d044c9bb14ceb8d637977966630b9 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20260204 08:52:49.796048 1747 tablet_bootstrap.cc:492] T 350b048c73bd4208a7aa70b3521869e1 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:49.797931 1747 tablet_bootstrap.cc:654] T 350b048c73bd4208a7aa70b3521869e1 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:49.803517 1745 tablet_bootstrap.cc:492] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:49.803737 1745 ts_tablet_manager.cc:1403] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.000s
I20260204 08:52:49.804790 1745 raft_consensus.cc:359] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.805028 1745 raft_consensus.cc:385] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.806469 1747 tablet_bootstrap.cc:492] T 350b048c73bd4208a7aa70b3521869e1 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:49.839120 1747 ts_tablet_manager.cc:1403] T 350b048c73bd4208a7aa70b3521869e1 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.043s user 0.000s sys 0.006s
I20260204 08:52:49.805116 1745 raft_consensus.cc:740] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:49.782078 1744 consensus_queue.cc:260] T e31d044c9bb14ceb8d637977966630b9 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.776237 1766 ts_tablet_manager.cc:1929] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260204 08:52:49.839671 1745 consensus_queue.cc:260] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.839825 1766 log.cc:1199] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/7121dc10d79548169242f5b65331c08a
I20260204 08:52:49.840009 1747 raft_consensus.cc:359] T 350b048c73bd4208a7aa70b3521869e1 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.840224 1747 raft_consensus.cc:385] T 350b048c73bd4208a7aa70b3521869e1 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.840279 1745 ts_tablet_manager.cc:1434] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.036s user 0.003s sys 0.000s
I20260204 08:52:49.840339 1766 ts_tablet_manager.cc:1950] T 7121dc10d79548169242f5b65331c08a P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:52:49.840370 1747 raft_consensus.cc:740] T 350b048c73bd4208a7aa70b3521869e1 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:49.840708 1745 tablet_bootstrap.cc:492] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:49.840619 1747 consensus_queue.cc:260] T 350b048c73bd4208a7aa70b3521869e1 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.841142 1747 ts_tablet_manager.cc:1434] T 350b048c73bd4208a7aa70b3521869e1 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260204 08:52:49.796402 1606 consensus_queue.cc:1048] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:49.842525 1745 tablet_bootstrap.cc:654] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:49.857241 1744 ts_tablet_manager.cc:1434] T e31d044c9bb14ceb8d637977966630b9 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.078s user 0.000s sys 0.014s
I20260204 08:52:49.858389 1745 tablet_bootstrap.cc:492] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:49.858572 1745 ts_tablet_manager.cc:1403] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.003s
I20260204 08:52:49.859426 1745 raft_consensus.cc:359] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.859637 1745 raft_consensus.cc:385] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.859728 1745 raft_consensus.cc:740] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:49.859968 1745 consensus_queue.cc:260] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.860718 1745 ts_tablet_manager.cc:1434] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260204 08:52:49.861035 1745 tablet_bootstrap.cc:492] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:49.863193 1745 tablet_bootstrap.cc:654] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:49.865423 1744 tablet_bootstrap.cc:492] T 350b048c73bd4208a7aa70b3521869e1 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:49.867399 1744 tablet_bootstrap.cc:654] T 350b048c73bd4208a7aa70b3521869e1 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:49.868487 1787 consensus_queue.cc:1048] T cb72d1554d9b4c33b6cd7c01fad765db P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:49.869642 1745 tablet_bootstrap.cc:492] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:49.869810 1745 ts_tablet_manager.cc:1403] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.001s
I20260204 08:52:49.870616 1745 raft_consensus.cc:359] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.870826 1745 raft_consensus.cc:385] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.870899 1745 raft_consensus.cc:740] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:49.900070 1744 tablet_bootstrap.cc:492] T 350b048c73bd4208a7aa70b3521869e1 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:49.900336 1744 ts_tablet_manager.cc:1403] T 350b048c73bd4208a7aa70b3521869e1 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.035s user 0.003s sys 0.021s
I20260204 08:52:49.871217 1745 consensus_queue.cc:260] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.905825 1745 ts_tablet_manager.cc:1434] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.036s user 0.002s sys 0.000s
I20260204 08:52:49.901252 1744 raft_consensus.cc:359] T 350b048c73bd4208a7aa70b3521869e1 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.906339 1734 raft_consensus.cc:493] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:49.906499 1734 raft_consensus.cc:515] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.906244 1744 raft_consensus.cc:385] T 350b048c73bd4208a7aa70b3521869e1 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:49.906672 1744 raft_consensus.cc:740] T 350b048c73bd4208a7aa70b3521869e1 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:49.906955 1744 consensus_queue.cc:260] T 350b048c73bd4208a7aa70b3521869e1 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.907225 1734 leader_election.cc:290] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:49.907519 1744 ts_tablet_manager.cc:1434] T 350b048c73bd4208a7aa70b3521869e1 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.007s user 0.000s sys 0.001s
I20260204 08:52:49.910138 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "350b048c73bd4208a7aa70b3521869e1" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:49.910377 1296 raft_consensus.cc:2468] T 350b048c73bd4208a7aa70b3521869e1 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:49.910862 1096 leader_election.cc:304] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:49.911271 1020 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "350b048c73bd4208a7aa70b3521869e1" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:49.911489 1020 raft_consensus.cc:2468] T 350b048c73bd4208a7aa70b3521869e1 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:49.911756 1733 raft_consensus.cc:2804] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:49.911860 1733 raft_consensus.cc:493] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:49.911935 1733 raft_consensus.cc:3060] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.915290 1733 raft_consensus.cc:515] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.915922 1733 leader_election.cc:290] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Requested vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:49.916144 1787 raft_consensus.cc:493] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:49.916360 1787 raft_consensus.cc:515] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.916548 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "350b048c73bd4208a7aa70b3521869e1" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:49.916721 1296 raft_consensus.cc:3060] T 350b048c73bd4208a7aa70b3521869e1 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.916976 1787 leader_election.cc:290] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:49.917680 1160 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "52abcf65cd4840fdb7719ada0979c7a1" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:49.917877 1160 raft_consensus.cc:2468] T 52abcf65cd4840fdb7719ada0979c7a1 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 0.
I20260204 08:52:49.918308 958 leader_election.cc:304] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:49.918605 1787 raft_consensus.cc:2804] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:49.918684 1787 raft_consensus.cc:493] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:49.918792 1787 raft_consensus.cc:3060] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.919427 1296 raft_consensus.cc:2468] T 350b048c73bd4208a7aa70b3521869e1 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:49.919766 1020 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "350b048c73bd4208a7aa70b3521869e1" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:49.919898 1295 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "52abcf65cd4840fdb7719ada0979c7a1" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:49.919974 1020 raft_consensus.cc:3060] T 350b048c73bd4208a7aa70b3521869e1 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.920101 1295 raft_consensus.cc:2468] T 52abcf65cd4840fdb7719ada0979c7a1 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 0.
I20260204 08:52:49.920099 1096 leader_election.cc:304] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:49.922514 1787 raft_consensus.cc:515] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.922782 1020 raft_consensus.cc:2468] T 350b048c73bd4208a7aa70b3521869e1 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:49.923576 1733 raft_consensus.cc:2804] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:49.923916 1160 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "52abcf65cd4840fdb7719ada0979c7a1" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:49.924115 1160 raft_consensus.cc:3060] T 52abcf65cd4840fdb7719ada0979c7a1 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.924346 1733 raft_consensus.cc:697] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Becoming Leader. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Running, Role: LEADER
I20260204 08:52:49.924713 1733 consensus_queue.cc:237] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.927070 1160 raft_consensus.cc:2468] T 52abcf65cd4840fdb7719ada0979c7a1 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 1.
I20260204 08:52:49.928681 958 leader_election.cc:304] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:49.929111 1606 raft_consensus.cc:2804] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:49.929311 1606 raft_consensus.cc:697] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 LEADER]: Becoming Leader. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Running, Role: LEADER
I20260204 08:52:49.929592 1606 consensus_queue.cc:237] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.932183 1787 leader_election.cc:290] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 election: Requested vote from peers fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:49.932330 1295 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "52abcf65cd4840fdb7719ada0979c7a1" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:49.932567 1295 raft_consensus.cc:3060] T 52abcf65cd4840fdb7719ada0979c7a1 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.935700 1295 raft_consensus.cc:2468] T 52abcf65cd4840fdb7719ada0979c7a1 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 1.
I20260204 08:52:49.962819 1538 raft_consensus.cc:493] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:49.963009 1538 raft_consensus.cc:515] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.963708 1538 leader_election.cc:290] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:49.964392 1020 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "822cc58f9ac042bf8b21c8ac3fc30ec2" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:49.964627 1020 raft_consensus.cc:2468] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:49.964540 1160 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "822cc58f9ac042bf8b21c8ac3fc30ec2" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:49.964879 1160 raft_consensus.cc:2468] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 0.
I20260204 08:52:49.965094 1232 leader_election.cc:304] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:49.965442 1538 raft_consensus.cc:2804] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:49.965567 1538 raft_consensus.cc:493] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:49.965653 1538 raft_consensus.cc:3060] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.968357 1538 raft_consensus.cc:515] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.969110 1538 leader_election.cc:290] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:49.969682 1160 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "822cc58f9ac042bf8b21c8ac3fc30ec2" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:49.969699 1020 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "822cc58f9ac042bf8b21c8ac3fc30ec2" candidate_uuid: "1234acd7d0db405085a5d95795007134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:49.969906 1020 raft_consensus.cc:3060] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.969907 1160 raft_consensus.cc:3060] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.971813 1160 raft_consensus.cc:2468] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:49.972285 1232 leader_election.cc:304] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:49.972632 1538 raft_consensus.cc:2804] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:49.972710 1020 raft_consensus.cc:2468] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1234acd7d0db405085a5d95795007134 in term 1.
I20260204 08:52:49.972776 1538 raft_consensus.cc:697] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Becoming Leader. State: Replica: 1234acd7d0db405085a5d95795007134, State: Running, Role: LEADER
I20260204 08:52:49.973299 1538 consensus_queue.cc:237] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:49.989621 1733 raft_consensus.cc:493] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:49.989856 1733 raft_consensus.cc:515] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.990552 1733 leader_election.cc:290] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:49.991102 1295 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e31d044c9bb14ceb8d637977966630b9" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:49.991176 1020 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e31d044c9bb14ceb8d637977966630b9" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:49.991366 1295 raft_consensus.cc:2468] T e31d044c9bb14ceb8d637977966630b9 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:49.991446 1020 raft_consensus.cc:2468] T e31d044c9bb14ceb8d637977966630b9 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:49.991966 1096 leader_election.cc:304] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:49.992424 1733 raft_consensus.cc:2804] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:49.992561 1733 raft_consensus.cc:493] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:49.992668 1733 raft_consensus.cc:3060] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.994691 1733 raft_consensus.cc:515] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
I20260204 08:52:49.995357 1733 leader_election.cc:290] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Requested vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:49.995862 1295 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e31d044c9bb14ceb8d637977966630b9" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:49.995918 1020 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e31d044c9bb14ceb8d637977966630b9" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:49.996110 1295 raft_consensus.cc:3060] T e31d044c9bb14ceb8d637977966630b9 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.996138 1020 raft_consensus.cc:3060] T e31d044c9bb14ceb8d637977966630b9 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:49.999027 1295 raft_consensus.cc:2468] T e31d044c9bb14ceb8d637977966630b9 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:49.999030 1020 raft_consensus.cc:2468] T e31d044c9bb14ceb8d637977966630b9 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:49.999497 1096 leader_election.cc:304] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:49.999960 1733 raft_consensus.cc:2804] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:50.000128 1733 raft_consensus.cc:697] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Becoming Leader. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Running, Role: LEADER
I20260204 08:52:50.000427 1733 consensus_queue.cc:237] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } }
W20260204 08:52:50.010514 1094 connection.cc:537] server connection from 127.0.0.1:57288 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:52:50.010535 956 connection.cc:537] server connection from 127.0.0.1:33412 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:52:50.011045 1351 connection.cc:537] client connection to 127.0.104.188:38981 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:52:50.011299 748 connection.cc:537] server connection from 127.0.0.1:41800 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:52:50.012100 1351 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.104.188:38981)
W20260204 08:52:50.012976 1351 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.104.188:38981)
W20260204 08:52:50.013727 1351 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.104.188:38981)
W20260204 08:52:50.013862 955 connection.cc:537] client connection to 127.0.104.188:38981 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:52:50.014468 1093 connection.cc:537] client connection to 127.0.104.188:38981 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:52:50.014838 1511 connection.cc:537] server connection from 127.0.0.1:33678 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:52:50.015199 1231 connection.cc:537] server connection from 127.0.0.1:36752 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260204 08:52:50.011492 1460 master_runner.cc:186] Error getting master registration for 127.0.104.188:38981: Network error: recv got EOF from 127.0.104.188:38981 (error 108),
I20260204 08:52:50.016414 1460 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:52:50.188242 1803 raft_consensus.cc:493] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c1e719e457e94a5da71929720411ae69)
I20260204 08:52:50.188482 1803 raft_consensus.cc:515] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:50.189121 1803 leader_election.cc:290] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
I20260204 08:52:50.189710 1592 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d083ecc9895345e5b06cd00ef0246c71" candidate_term: 3 candidate_status { last_received { term: 2 index: 91 } } ignore_live_leader: false dest_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" is_pre_election: true
I20260204 08:52:50.189926 1592 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d083ecc9895345e5b06cd00ef0246c71 in term 2.
I20260204 08:52:50.190446 747 leader_election.cc:304] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d083ecc9895345e5b06cd00ef0246c71, ebeeb5d4823b41b3a8e37546ff2f6e34; no voters:
I20260204 08:52:50.190713 1803 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260204 08:52:50.190855 1803 raft_consensus.cc:493] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c1e719e457e94a5da71929720411ae69)
I20260204 08:52:50.190958 1803 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 2 FOLLOWER]: Advancing to term 3
W20260204 08:52:50.191273 746 leader_election.cc:336] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:52:50.192958 1803 raft_consensus.cc:515] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:50.193456 1803 leader_election.cc:290] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 3 election: Requested vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
I20260204 08:52:50.193948 1592 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d083ecc9895345e5b06cd00ef0246c71" candidate_term: 3 candidate_status { last_received { term: 2 index: 91 } } ignore_live_leader: false dest_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34"
I20260204 08:52:50.194164 1592 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 2 FOLLOWER]: Advancing to term 3
W20260204 08:52:50.194733 746 leader_election.cc:336] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:52:50.196245 1592 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d083ecc9895345e5b06cd00ef0246c71 in term 3.
I20260204 08:52:50.196715 747 leader_election.cc:304] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d083ecc9895345e5b06cd00ef0246c71, ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69
I20260204 08:52:50.197090 1803 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 3 FOLLOWER]: Leader election won for term 3
I20260204 08:52:50.197722 1803 raft_consensus.cc:697] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 3 LEADER]: Becoming Leader. State: Replica: d083ecc9895345e5b06cd00ef0246c71, State: Running, Role: LEADER
I20260204 08:52:50.198011 1803 consensus_queue.cc:237] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 90, Committed index: 90, Last appended: 2.91, Last appended by leader: 91, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:50.199489 1806 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d083ecc9895345e5b06cd00ef0246c71. Latest consensus state: current_term: 3 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:50.199791 1806 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: LEADER
I20260204 08:52:50.270309 1733 consensus_queue.cc:1048] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:50.278865 1733 consensus_queue.cc:1048] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:50.288570 418 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu pid 1460
I20260204 08:52:50.295558 1624 generic_service.cc:196] Checking for leaks (request via RPC)
I20260204 08:52:50.410538 418 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu with pid 1460
I20260204 08:52:50.599064 1538 consensus_queue.cc:1048] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:50.603583 1160 raft_consensus.cc:3060] T 0771de581fcb4c90823dfaa8bf339cc9 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:50.608629 1817 consensus_queue.cc:1048] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260204 08:52:50.627020 746 consensus_peers.cc:597] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 -> Peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Couldn't send request to peer c1e719e457e94a5da71929720411ae69. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260204 08:52:50.662212 747 consensus_peers.cc:597] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 -> Peer ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471): Couldn't send request to peer ebeeb5d4823b41b3a8e37546ff2f6e34. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.190:42471: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260204 08:52:50.741660 1810 consensus_queue.cc:1048] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:50.748605 1810 consensus_queue.cc:1048] T 350b048c73bd4208a7aa70b3521869e1 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:50.895695 1606 consensus_queue.cc:1048] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:50.915748 1829 consensus_queue.cc:1048] T 5135d9b498314d9098339db68f64207d P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:51.056844 1817 consensus_queue.cc:1048] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:51.057380 1810 consensus_queue.cc:1048] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:51.064674 1818 consensus_queue.cc:1048] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:51.066205 1810 consensus_queue.cc:1048] T e31d044c9bb14ceb8d637977966630b9 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:51.165786 1829 consensus_queue.cc:1048] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:51.172405 1832 consensus_queue.cc:1048] T 52abcf65cd4840fdb7719ada0979c7a1 P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260204 08:52:51.206652 1807 op_tracker.cc:284] OpTracker waiting for 1 outstanding ops to complete now for 1.006s
I20260204 08:52:51.206789 1807 op_tracker.cc:286] Dumping up to 50 currently running ops:
I20260204 08:52:51.206871 1807 op_tracker.cc:290] R-P WriteOp [type=FOLLOWER, start_time=2026-02-04 08:52:49, state=WriteOpState 0x617000176c00 [op_id=(term: 2 index: 91), ts=7250719415032356864, rows=[INSERT (int8 entry_type=<redacted>, string entry_id=<redacted>, string metadata=<redacted>), MUTATE (int8 entry_type=<redacted>, string entry_id=<redacted>) SET metadata=<redacted>]]]
W20260204 08:52:51.226854 749 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 96ed25c622d946dfb4531ae603a80bad (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 96ed25c622d946dfb4531ae603a80bad consensus state has no leader
I20260204 08:52:51.288697 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
/tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:42471
--webserver_interface=127.0.104.190
--webserver_port=40787
--builtin_ntp_servers=127.0.104.148:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260204 08:52:51.647611 1846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260204 08:52:51.647935 1846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260204 08:52:51.648042 1846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260204 08:52:51.656620 1846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260204 08:52:51.656757 1846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260204 08:52:51.656839 1846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260204 08:52:51.656894 1846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260204 08:52:51.667912 1846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:42471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.104.190
--webserver_port=40787
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0efbebd487e7d0c7f21ab0d5a91e0fa70046af83
build type FASTDEBUG
built by None at 04 Feb 2026 08:43:20 UTC on 24a791456cd2
build id 10340
ASAN enabled
I20260204 08:52:51.669667 1846 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260204 08:52:51.671604 1846 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260204 08:52:51.681054 1854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:51.681162 1853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:51.681653 1856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260204 08:52:51.682296 1846 server_base.cc:1060] running on GCE node
I20260204 08:52:51.683250 1846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260204 08:52:51.685112 1846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260204 08:52:51.686415 1846 hybrid_clock.cc:648] HybridClock initialized: now 1770195171686336 us; error 62 us; skew 500 ppm
I20260204 08:52:51.686889 1846 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260204 08:52:51.689281 1846 webserver.cc:492] Webserver started at http://127.0.104.190:40787/ using document root <none> and password file <none>
I20260204 08:52:51.690053 1846 fs_manager.cc:362] Metadata directory not provided
I20260204 08:52:51.690150 1846 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260204 08:52:51.695859 1846 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260204 08:52:51.699334 1862 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260204 08:52:51.700881 1846 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260204 08:52:51.701037 1846 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data,/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34"
format_stamp: "Formatted at 2026-02-04 08:52:41 on dist-test-slave-vp4h"
I20260204 08:52:51.701741 1846 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260204 08:52:51.718778 1846 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260204 08:52:51.719630 1846 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260204 08:52:51.719910 1846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260204 08:52:51.740417 1846 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.190:42471
I20260204 08:52:51.740463 1913 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.190:42471 every 8 connection(s)
I20260204 08:52:51.742198 1846 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
I20260204 08:52:51.747372 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu as pid 1846
I20260204 08:52:51.751036 1914 sys_catalog.cc:263] Verifying existing consensus state
I20260204 08:52:51.756032 1914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Bootstrap starting.
I20260204 08:52:51.772648 1205 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:52:51.777475 1914 log.cc:826] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Log is configured to *not* fsync() on all Append() calls
I20260204 08:52:51.842310 1914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Bootstrap replayed 1/1 log segments. Stats: ops{read=89 overwritten=0 applied=88 ignored=0} inserts{seen=60 ignored=0} mutations{seen=75 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260204 08:52:51.842962 1914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Bootstrap complete.
I20260204 08:52:51.848188 1914 raft_consensus.cc:359] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 3 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:51.850941 1914 raft_consensus.cc:740] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebeeb5d4823b41b3a8e37546ff2f6e34, State: Initialized, Role: FOLLOWER
I20260204 08:52:51.851713 1914 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 88, Last appended: 2.89, Last appended by leader: 89, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:51.852509 1920 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:51.852756 1920 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:51.853551 1914 sys_catalog.cc:565] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: configured and running, proceeding with master startup.
I20260204 08:52:51.866611 1067 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:52:51.874414 1878 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" instance_seqno: 1770195163347042) as {username='slave'} at 127.0.104.129:37529; Asking this server to re-register.
I20260204 08:52:51.875612 1067 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:51.878340 1878 ts_manager.cc:194] Registered new tserver with Master: 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:51.879376 1937 catalog_manager.cc:1269] Loaded cluster ID: 41c7773c62174f70835c9cfa744e1148
I20260204 08:52:51.879468 1937 catalog_manager.cc:1562] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: loading cluster ID for follower catalog manager: success
I20260204 08:52:51.885295 1937 catalog_manager.cc:1584] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: acquiring CA information for follower catalog manager: success
I20260204 08:52:51.889065 1937 catalog_manager.cc:1612] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260204 08:52:51.909997 1846 master_runner.cc:186] Error getting master registration for 127.0.104.188:38981: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111),
I20260204 08:52:51.915573 1846 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:52:51.973519 1341 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:52:51.975665 1879 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1234acd7d0db405085a5d95795007134" instance_seqno: 1770195164422899) as {username='slave'} at 127.0.104.131:32827; Asking this server to re-register.
I20260204 08:52:51.976555 1341 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:51.978549 1879 ts_manager.cc:194] Registered new tserver with Master: 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:52.166503 1879 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" instance_seqno: 1770195163874581) as {username='slave'} at 127.0.104.130:37323; Asking this server to re-register.
I20260204 08:52:52.167289 1205 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:52.169198 1879 ts_manager.cc:194] Registered new tserver with Master: fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:52.229079 1889 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 3 FOLLOWER]: Refusing update from remote peer d083ecc9895345e5b06cd00ef0246c71: Log matching property violated. Preceding OpId in replica: term: 2 index: 89. Preceding OpId from leader: term: 2 index: 91. (index mismatch)
I20260204 08:52:52.230108 1843 consensus_queue.cc:1050] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 92, Last known committed idx: 88, Time since last communication: 0.000s
I20260204 08:52:52.236114 1920 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d083ecc9895345e5b06cd00ef0246c71. Latest consensus state: current_term: 3 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:52.236430 1920 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:52.238873 1920 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:52.239127 1920 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:52.241313 1944 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:52.241943 1944 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: LEADER
I20260204 08:52:52.433370 728 master_runner.cc:186] Error getting master registration for 127.0.104.188:38981: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111),
I20260204 08:52:52.433559 728 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260204 08:52:52.457494 746 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111) [suppressed 9 similar messages]
I20260204 08:52:52.654752 1807 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260204 08:52:52.739298 1920 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:52:52.739634 1920 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:52.742038 1920 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:52:52.743034 1863 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:52:52.744856 1863 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:52:52.861128 746 consensus_peers.cc:597] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 -> Peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Couldn't send request to peer c1e719e457e94a5da71929720411ae69. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260204 08:52:54.129060 1865 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Timed out: RequestConsensusVote RPC to 127.0.104.189:37005 timed out after 0.692s (SENT)
I20260204 08:52:54.129268 1865 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:52:54.130128 1968 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260204 08:52:54.275966 418 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu pid 728
I20260204 08:52:54.277393 792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260204 08:52:54.414619 418 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu with pid 728
W20260204 08:52:54.435829 1864 connection.cc:537] server connection from 127.0.0.1:34150 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260204 08:52:54.916921 1978 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:52:54.917248 1978 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:54.918107 1978 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:52:54.922066 1865 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:52:54.922322 1863 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:52:54.922451 1863 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:52:54.922725 1978 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260204 08:52:55.294317 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
/tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:37005
--webserver_interface=127.0.104.189
--webserver_port=43227
--builtin_ntp_servers=127.0.104.148:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260204 08:52:55.538666 1994 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:52:55.538894 1994 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:55.539705 1994 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:52:55.545768 1863 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:52:55.548413 1865 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:52:55.548597 1865 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:52:55.549096 1994 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260204 08:52:55.658882 1989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260204 08:52:55.659245 1989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260204 08:52:55.659363 1989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260204 08:52:55.668460 1989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260204 08:52:55.668560 1989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260204 08:52:55.668623 1989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260204 08:52:55.668684 1989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260204 08:52:55.680337 1989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:37005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.104.189
--webserver_port=43227
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0efbebd487e7d0c7f21ab0d5a91e0fa70046af83
build type FASTDEBUG
built by None at 04 Feb 2026 08:43:20 UTC on 24a791456cd2
build id 10340
ASAN enabled
I20260204 08:52:55.682407 1989 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260204 08:52:55.684350 1989 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260204 08:52:55.692839 2004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:55.692879 2005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260204 08:52:55.696612 1989 server_base.cc:1060] running on GCE node
W20260204 08:52:55.697582 2007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260204 08:52:55.698529 1989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260204 08:52:55.700134 1989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260204 08:52:55.701392 1989 hybrid_clock.cc:648] HybridClock initialized: now 1770195175701336 us; error 68 us; skew 500 ppm
I20260204 08:52:55.701781 1989 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260204 08:52:55.704197 1989 webserver.cc:492] Webserver started at http://127.0.104.189:43227/ using document root <none> and password file <none>
I20260204 08:52:55.705024 1989 fs_manager.cc:362] Metadata directory not provided
I20260204 08:52:55.705152 1989 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260204 08:52:55.711233 1989 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260204 08:52:55.715191 2013 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260204 08:52:55.716904 1989 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260204 08:52:55.717114 1989 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data,/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
uuid: "d083ecc9895345e5b06cd00ef0246c71"
format_stamp: "Formatted at 2026-02-04 08:52:41 on dist-test-slave-vp4h"
I20260204 08:52:55.717861 1989 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260204 08:52:55.750345 1989 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260204 08:52:55.751369 1989 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260204 08:52:55.751665 1989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260204 08:52:55.771718 1989 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.189:37005
I20260204 08:52:55.771752 2064 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.189:37005 every 8 connection(s)
I20260204 08:52:55.773540 1989 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data/info.pb
I20260204 08:52:55.782011 2065 sys_catalog.cc:263] Verifying existing consensus state
I20260204 08:52:55.783923 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu as pid 1989
I20260204 08:52:55.786168 2065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Bootstrap starting.
I20260204 08:52:55.808519 2065 log.cc:826] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Log is configured to *not* fsync() on all Append() calls
I20260204 08:52:55.880160 2065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Bootstrap replayed 1/1 log segments. Stats: ops{read=92 overwritten=0 applied=92 ignored=0} inserts{seen=63 ignored=0} mutations{seen=77 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260204 08:52:55.880739 2065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Bootstrap complete.
I20260204 08:52:55.887573 2065 raft_consensus.cc:359] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:55.888624 2065 raft_consensus.cc:740] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: d083ecc9895345e5b06cd00ef0246c71, State: Initialized, Role: FOLLOWER
I20260204 08:52:55.889462 2065 consensus_queue.cc:260] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 92, Last appended: 3.92, Last appended by leader: 92, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:55.890466 2070 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:55.890856 2070 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:55.892272 2065 sys_catalog.cc:565] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: configured and running, proceeding with master startup.
I20260204 08:52:55.915025 2084 catalog_manager.cc:1269] Loaded cluster ID: 41c7773c62174f70835c9cfa744e1148
I20260204 08:52:55.915161 2084 catalog_manager.cc:1562] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: loading cluster ID for follower catalog manager: success
I20260204 08:52:55.920228 2084 catalog_manager.cc:1584] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: acquiring CA information for follower catalog manager: success
I20260204 08:52:55.924907 2084 catalog_manager.cc:1612] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260204 08:52:55.932988 1989 master_runner.cc:186] Error getting master registration for 127.0.104.188:38981: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111),
I20260204 08:52:55.936728 1989 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:52:55.953199 1069 heartbeater.cc:344] Connected to a master server at 127.0.104.189:37005
I20260204 08:52:55.956604 2030 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" instance_seqno: 1770195163347042) as {username='slave'} at 127.0.104.129:35545; Asking this server to re-register.
I20260204 08:52:55.957525 1069 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:55.959820 2030 ts_manager.cc:194] Registered new tserver with Master: 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:56.004345 1343 heartbeater.cc:344] Connected to a master server at 127.0.104.189:37005
I20260204 08:52:56.006832 2030 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1234acd7d0db405085a5d95795007134" instance_seqno: 1770195164422899) as {username='slave'} at 127.0.104.131:48471; Asking this server to re-register.
I20260204 08:52:56.007591 1343 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:56.009677 2030 ts_manager.cc:194] Registered new tserver with Master: 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:56.186218 1207 heartbeater.cc:344] Connected to a master server at 127.0.104.189:37005
I20260204 08:52:56.187700 2030 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" instance_seqno: 1770195163874581) as {username='slave'} at 127.0.104.130:59535; Asking this server to re-register.
I20260204 08:52:56.188289 1207 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:56.189968 2030 ts_manager.cc:194] Registered new tserver with Master: fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:56.477864 2095 raft_consensus.cc:493] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:56.478236 2095 raft_consensus.cc:515] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:56.480340 2095 leader_election.cc:290] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
I20260204 08:52:56.480937 1889 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d083ecc9895345e5b06cd00ef0246c71" candidate_term: 4 candidate_status { last_received { term: 3 index: 92 } } ignore_live_leader: false dest_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" is_pre_election: true
I20260204 08:52:56.481325 1889 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d083ecc9895345e5b06cd00ef0246c71 in term 3.
W20260204 08:52:56.481353 2014 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:52:56.481982 2015 leader_election.cc:304] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d083ecc9895345e5b06cd00ef0246c71, ebeeb5d4823b41b3a8e37546ff2f6e34; no voters:
I20260204 08:52:56.482473 2095 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260204 08:52:56.482636 2095 raft_consensus.cc:493] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:56.482760 2095 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 3 FOLLOWER]: Advancing to term 4
W20260204 08:52:56.482955 2014 leader_election.cc:336] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:52:56.486270 2095 raft_consensus.cc:515] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:56.486904 2095 leader_election.cc:290] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 4 election: Requested vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
I20260204 08:52:56.487401 1889 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d083ecc9895345e5b06cd00ef0246c71" candidate_term: 4 candidate_status { last_received { term: 3 index: 92 } } ignore_live_leader: false dest_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34"
I20260204 08:52:56.487641 1889 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 3 FOLLOWER]: Advancing to term 4
W20260204 08:52:56.488205 2014 leader_election.cc:336] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:52:56.490964 1889 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d083ecc9895345e5b06cd00ef0246c71 in term 4.
I20260204 08:52:56.491477 2015 leader_election.cc:304] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d083ecc9895345e5b06cd00ef0246c71, ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69
I20260204 08:52:56.491783 2095 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 4 FOLLOWER]: Leader election won for term 4
I20260204 08:52:56.492153 2095 raft_consensus.cc:697] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 4 LEADER]: Becoming Leader. State: Replica: d083ecc9895345e5b06cd00ef0246c71, State: Running, Role: LEADER
I20260204 08:52:56.492691 2095 consensus_queue.cc:237] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 92, Committed index: 92, Last appended: 3.92, Last appended by leader: 92, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:56.495604 2098 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d083ecc9895345e5b06cd00ef0246c71. Latest consensus state: current_term: 4 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:56.495880 2098 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: LEADER
I20260204 08:52:56.496783 2100 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260204 08:52:56.499780 2100 catalog_manager.cc:679] Loaded metadata for table default.table_f388b768c4b14290a90dd29962cc3fc4 [id=105a9508f80c4190a40ab4ab66af2872]
I20260204 08:52:56.500231 2100 catalog_manager.cc:679] Loaded metadata for table default.table_a638f2d4f2654854afdb0b065ee83f48 [id=12e17f7b128b4db99ae9041dba0602b5]
I20260204 08:52:56.500649 2100 catalog_manager.cc:679] Loaded metadata for table default.table_7bcbc3cfbc41406cba3ed79125a46e82 [id=180667e663144b7ea79638946d6c76cc]
I20260204 08:52:56.501001 2100 catalog_manager.cc:679] Loaded metadata for table default.table_e7fe420574dc40cdb70c4602f5a6e0db [id=19930831c99540688fb9a1e6ebfd8f82]
I20260204 08:52:56.501327 2100 catalog_manager.cc:679] Loaded metadata for table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]
I20260204 08:52:56.501652 2100 catalog_manager.cc:679] Loaded metadata for table default.table_97ec63740f2b4928af12e35b5eaa29cd [id=2a58ed39876b46aba637d263234e55d9]
I20260204 08:52:56.501968 2100 catalog_manager.cc:679] Loaded metadata for table default.table_f8273ef1a407421085918a8e3827f5f6 [id=2df47e721cd642e4b6fd5e61e172749c]
I20260204 08:52:56.502442 2100 catalog_manager.cc:679] Loaded metadata for table default.table_73e876d60c444a5daa316e4bf7158614 [id=42a606fb50bf4c58be67f67afab90d53]
I20260204 08:52:56.502734 2100 catalog_manager.cc:679] Loaded metadata for table default.table_1389d07a897b48439a706dce8abe591b [id=5a415846f1274434994325bb65b31911]
I20260204 08:52:56.503060 2100 catalog_manager.cc:679] Loaded metadata for table default.table_c97013dd16f84d8ba7cd528295f5ad49 [id=5f9f5b0850774ff9b8ecb93e141ff768]
I20260204 08:52:56.503350 2100 catalog_manager.cc:679] Loaded metadata for table default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807]
I20260204 08:52:56.503708 2100 catalog_manager.cc:679] Loaded metadata for table default.table_b4244312363c4346b110598eb01428a3 [id=7d4edb505f7940668b492a9d592a3e61]
I20260204 08:52:56.504041 2100 catalog_manager.cc:679] Loaded metadata for table default.table_9f93a995778f4fe6a4667cff0ef494b9 [id=893f125488f745a5b0bc9ec3d07da00b]
I20260204 08:52:56.504388 2100 catalog_manager.cc:679] Loaded metadata for table default.table_4f900fd574c849aba171e6a64bc43dbd [id=9427e77d6a3842a7a2665d99c8ebb99e]
I20260204 08:52:56.504705 2100 catalog_manager.cc:679] Loaded metadata for table default.table_477defa30c2c40e99360b505ef988a80 [id=b1e79546138b4dcb9c5a68977af78bf0]
I20260204 08:52:56.505008 2100 catalog_manager.cc:679] Loaded metadata for table default.table_f087faf331654fae99b16adcfb5f705f [id=b3b4145c845b417aa7582e41fb48b564]
I20260204 08:52:56.505460 2100 catalog_manager.cc:679] Loaded metadata for table default.table_9e950ede78084ef18f4e1051d9a42a90 [id=c626dc1c029e4b59902b4493f4e16a78]
I20260204 08:52:56.505815 2100 catalog_manager.cc:679] Loaded metadata for table default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294]
I20260204 08:52:56.506117 2100 catalog_manager.cc:679] Loaded metadata for table default.table_780ed695e5324a469614b35852a643cf [id=f29a6ab746244116a4815a632194b3ec]
I20260204 08:52:56.508342 2100 tablet_loader.cc:96] loaded metadata for tablet 043cdafd4b11463bae674695e7a74854 (table default.table_9f93a995778f4fe6a4667cff0ef494b9 [id=893f125488f745a5b0bc9ec3d07da00b])
I20260204 08:52:56.508713 2100 tablet_loader.cc:96] loaded metadata for tablet 0771de581fcb4c90823dfaa8bf339cc9 (table default.table_1389d07a897b48439a706dce8abe591b [id=5a415846f1274434994325bb65b31911])
I20260204 08:52:56.509167 2100 tablet_loader.cc:96] loaded metadata for tablet 2de648ce42af4a2fbd494c8cc3e38bb4 (table default.table_f388b768c4b14290a90dd29962cc3fc4 [id=105a9508f80c4190a40ab4ab66af2872])
I20260204 08:52:56.509472 2100 tablet_loader.cc:96] loaded metadata for tablet 350b048c73bd4208a7aa70b3521869e1 (table default.table_f8273ef1a407421085918a8e3827f5f6 [id=2df47e721cd642e4b6fd5e61e172749c])
I20260204 08:52:56.509752 2100 tablet_loader.cc:96] loaded metadata for tablet 35b597aac0264d36a44ae37c6ffb945e (table default.table_7bcbc3cfbc41406cba3ed79125a46e82 [id=180667e663144b7ea79638946d6c76cc])
I20260204 08:52:56.510164 2100 tablet_loader.cc:96] loaded metadata for tablet 4c26152dcc2147d6b0cb8d1242a7a73b (table default.table_477defa30c2c40e99360b505ef988a80 [id=b1e79546138b4dcb9c5a68977af78bf0])
I20260204 08:52:56.510432 2100 tablet_loader.cc:96] loaded metadata for tablet 5135d9b498314d9098339db68f64207d (table default.table_780ed695e5324a469614b35852a643cf [id=f29a6ab746244116a4815a632194b3ec])
I20260204 08:52:56.510681 2100 tablet_loader.cc:96] loaded metadata for tablet 52abcf65cd4840fdb7719ada0979c7a1 (table default.table_a638f2d4f2654854afdb0b065ee83f48 [id=12e17f7b128b4db99ae9041dba0602b5])
I20260204 08:52:56.511111 2100 tablet_loader.cc:96] loaded metadata for tablet 5fe7a46289d84efcb7b1455bd317d279 (table default.table_b4244312363c4346b110598eb01428a3 [id=7d4edb505f7940668b492a9d592a3e61])
I20260204 08:52:56.511943 2100 tablet_loader.cc:96] loaded metadata for tablet 80f72b647ee643afabcc5cafa9c43b13 (table default.table_9e950ede78084ef18f4e1051d9a42a90 [id=c626dc1c029e4b59902b4493f4e16a78])
I20260204 08:52:56.512331 2100 tablet_loader.cc:96] loaded metadata for tablet 81f37ac4f50a440d8e7292bfa05a0a7d (table default.table_73e876d60c444a5daa316e4bf7158614 [id=42a606fb50bf4c58be67f67afab90d53])
I20260204 08:52:56.512655 2100 tablet_loader.cc:96] loaded metadata for tablet 822cc58f9ac042bf8b21c8ac3fc30ec2 (table default.table_f087faf331654fae99b16adcfb5f705f [id=b3b4145c845b417aa7582e41fb48b564])
I20260204 08:52:56.513660 2100 tablet_loader.cc:96] loaded metadata for tablet c9ce4ab8ce1f48868b890df81e551eed (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421])
I20260204 08:52:56.513953 2100 tablet_loader.cc:96] loaded metadata for tablet cb72d1554d9b4c33b6cd7c01fad765db (table default.table_97ec63740f2b4928af12e35b5eaa29cd [id=2a58ed39876b46aba637d263234e55d9])
I20260204 08:52:56.514348 2100 tablet_loader.cc:96] loaded metadata for tablet d269b616e72f442e8a47f0c9192db8f8 (table default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807])
I20260204 08:52:56.514745 2100 tablet_loader.cc:96] loaded metadata for tablet deef771e18ac44a5b81f258fe8e6f3c4 (table default.table_4f900fd574c849aba171e6a64bc43dbd [id=9427e77d6a3842a7a2665d99c8ebb99e])
I20260204 08:52:56.515039 2100 tablet_loader.cc:96] loaded metadata for tablet e31d044c9bb14ceb8d637977966630b9 (table default.table_c97013dd16f84d8ba7cd528295f5ad49 [id=5f9f5b0850774ff9b8ecb93e141ff768])
I20260204 08:52:56.515317 2100 tablet_loader.cc:96] loaded metadata for tablet ebed8bb53e7947e08d0c960b13bacd8d (table default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294])
I20260204 08:52:56.515640 2100 tablet_loader.cc:96] loaded metadata for tablet fe0b2df895c643e7b69fb30a2b0b8e5c (table default.table_e7fe420574dc40cdb70c4602f5a6e0db [id=19930831c99540688fb9a1e6ebfd8f82])
I20260204 08:52:56.515933 2100 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260204 08:52:56.517539 2100 catalog_manager.cc:1269] Loaded cluster ID: 41c7773c62174f70835c9cfa744e1148
I20260204 08:52:56.517616 2100 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260204 08:52:56.519203 2100 catalog_manager.cc:1514] Loading token signing keys...
I20260204 08:52:56.520419 2100 catalog_manager.cc:6038] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Loaded TSK: 0
I20260204 08:52:56.521350 2100 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260204 08:52:56.522794 2100 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260204 08:52:56.524107 2100 catalog_manager.cc:5876] Last processed Hive Metastore notification event ID: 20
I20260204 08:52:56.549487 2029 master_service.cc:954] ReplaceTablet: received request to replace tablet 822cc58f9ac042bf8b21c8ac3fc30ec2 from {username='slave'} at 127.0.0.1:40830
I20260204 08:52:56.549705 2030 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40830:
name: "default.Table_14079082888e40fc818c5b295ee4da92"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:56.550078 2026 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40830:
name: "default.Table_27cce8dee4984465be213744abfadc9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:56.550638 2028 master_service.cc:954] ReplaceTablet: received request to replace tablet 0771de581fcb4c90823dfaa8bf339cc9 from {username='slave'} at 127.0.0.1:40830
I20260204 08:52:56.550781 2027 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40830:
name: "default.Table_2b92e77d55da4edb9ab442d799aa9e06"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:56.551925 2025 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40830:
name: "default.Table_85f07b4907b24f1ca63791592455b59c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:56.552714 2024 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40830:
name: "default.Table_8c0106d78fea4bd9846bde27656144cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:56.553836 2021 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40830:
table { table_name: "default.Table_97ec63740f2b4928af12e35b5eaa29cd" } modify_external_catalogs: true
I20260204 08:52:56.553781 2023 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40830:
name: "default.Table_8c3f05619b754ead910a58656d67b3b7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:56.554760 2021 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40830:
table { table_name: "default.Table_97ec63740f2b4928af12e35b5eaa29cd" } modify_external_catalogs: true
I20260204 08:52:56.554670 2022 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40830:
name: "default.Table_81228fe5292c485e8e2260554ca79fd7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:56.559922 1889 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 4 FOLLOWER]: Refusing update from remote peer d083ecc9895345e5b06cd00ef0246c71: Log matching property violated. Preceding OpId in replica: term: 3 index: 92. Preceding OpId from leader: term: 4 index: 95. (index mismatch)
F20260204 08:52:56.559970 2083 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 23, previous 23; dumping first 4 out of 15 received events: NotificationEvent(eventId=21, eventTime=1770195168, eventType=DROP_TABLE, dbName=default, tableName=table_85f07b4907b24f1ca63791592455b59c, message={"server":"","servicePrincipal":"","db":"default","table":"table_85f07b4907b24f1ca63791592455b59c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_85f07b4907b24f1ca63791592455b59c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195167},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_85f07b4907b24f1ca63791592455b59c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_85f07b4907b24f1ca63791592455b59c\",\"kudu.table_id\":\"158d366d5ccf4b78b73903776a30cb35\",\"transient_lastDdlTime\":\"1770195167\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=22, eventTime=1770195168, eventType=CREATE_TABLE, dbName=default, tableName=table_97ec63740f2b4928af12e35b5eaa29cd, message={"server":"","servicePrincipal":"","db":"default","table":"table_97ec63740f2b4928af12e35b5eaa29cd","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_97ec63740f2b4928af12e35b5eaa29cd\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195168},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_97ec63740f2b4928af12e35b5eaa29cd\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_97ec63740f2b4928af12e35b5eaa29cd\",\"kudu.table_id\":\"2a58ed39876b46aba637d263234e55d9\",\"transient_lastDdlTime\":\"1770195168\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=23, eventTime=1770195168, eventType=DROP_TABLE, dbName=default, tableName=table_279632cf491c4198ad9b91a114c3fc33, message={"server":"","servicePrincipal":"","db":"default","table":"table_279632cf491c4198ad9b91a114c3fc33","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_279632cf491c4198ad9b91a114c3fc33\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195165},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_279632cf491c4198ad9b91a114c3fc33\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_279632cf491c4198ad9b91a114c3fc33\",\"kudu.table_id\":\"da8943eba12e426ba3bac81a64141294\",\"transient_lastDdlTime\":\"1770195165\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=23, eventTime=1770195168, eventType=CREATE_TABLE, dbName=default, tableName=table_477defa30c2c40e99360b505ef988a80, message={"server":"","servicePrincipal":"","db":"default","table":"table_477defa30c2c40e99360b505ef988a80","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_477defa30c2c40e99360b505ef988a80\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195168},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_477defa30c2c40e99360b505ef988a80\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_477defa30c2c40e99360b505ef988a80\",\"kudu.table_id\":\"b1e79546138b4dcb9c5a68977af78bf0\",\"transient_lastDdlTime\":\"1770195168\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20260204 08:52:56.560907 2102 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 93, Last known committed idx: 92, Time since last communication: 0.000s
W20260204 08:52:56.561408 2014 consensus_peers.cc:597] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 -> Peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Couldn't send request to peer c1e719e457e94a5da71929720411ae69. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7f4f54507dcd google::LogMessage::Fail() at ??:0
@ 0x7f4f5450bb93 google::LogMessage::SendToLog() at ??:0
@ 0x7f4f545077cc google::LogMessage::Flush() at ??:0
I20260204 08:52:56.565287 2104 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d083ecc9895345e5b06cd00ef0246c71. Latest consensus state: current_term: 4 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:56.565546 2104 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f4f54507a11 google::LogMessage::~LogMessage() at ??:0
I20260204 08:52:56.568168 2101 mvcc.cc:204] Tried to move back new op lower bound from 7250719443174797312 to 7250719442917978112. Current Snapshot: MvccSnapshot[applied={T|T < 7250719443174797312}]
I20260204 08:52:56.571413 2028 catalog_manager.cc:5941] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Sending DeleteTablet for 3 replicas of tablet 0771de581fcb4c90823dfaa8bf339cc9
I20260204 08:52:56.572113 2029 catalog_manager.cc:5941] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Sending DeleteTablet for 3 replicas of tablet 822cc58f9ac042bf8b21c8ac3fc30ec2
I20260204 08:52:56.573930 2029 catalog_manager.cc:6724] ReplaceTablet: tablet 822cc58f9ac042bf8b21c8ac3fc30ec2 deleted and replaced by tablet f65514e5263e4f86a34195435040910f
I20260204 08:52:56.577756 2028 catalog_manager.cc:6724] ReplaceTablet: tablet 0771de581fcb4c90823dfaa8bf339cc9 deleted and replaced by tablet eda7388ec12f4a7386823dadedaf2e2f
@ 0x7f4f6af3b7f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f4f6af3a0a1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260204 08:52:56.603233 1000 tablet_service.cc:1552] Processing DeleteTablet for tablet 822cc58f9ac042bf8b21c8ac3fc30ec2 with delete_type TABLET_DATA_DELETED (replaced by tablet f65514e5263e4f86a34195435040910f) from {username='slave'} at 127.0.0.1:38438
I20260204 08:52:56.603255 1002 tablet_service.cc:1552] Processing DeleteTablet for tablet 0771de581fcb4c90823dfaa8bf339cc9 with delete_type TABLET_DATA_DELETED (replaced by tablet eda7388ec12f4a7386823dadedaf2e2f) from {username='slave'} at 127.0.0.1:38438
@ 0x7f4f6af41d1c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f4f6af41b99 std::_Function_handler<>::_M_invoke() at ??:0
I20260204 08:52:56.614389 2095 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:56.614670 2029 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40830:
table { table_name: "default.Table_477defa30c2c40e99360b505ef988a80" } modify_external_catalogs: true
I20260204 08:52:56.614974 2029 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40830:
table { table_name: "default.Table_477defa30c2c40e99360b505ef988a80" } modify_external_catalogs: true
@ 0xa8970d std::function<>::operator()() at ??:?
I20260204 08:52:56.617138 2104 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:56.617353 2104 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:56.618705 2117 tablet_replica.cc:333] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:52:56.618932 2117 raft_consensus.cc:2243] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:56.619167 2117 raft_consensus.cc:2272] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:56.619799 2117 ts_tablet_manager.cc:1916] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:56.619822 1139 tablet_service.cc:1552] Processing DeleteTablet for tablet 0771de581fcb4c90823dfaa8bf339cc9 with delete_type TABLET_DATA_DELETED (replaced by tablet eda7388ec12f4a7386823dadedaf2e2f) from {username='slave'} at 127.0.0.1:56370
I20260204 08:52:56.619815 2028 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40830:
table { table_name: "default.Table_7bcbc3cfbc41406cba3ed79125a46e82" } modify_external_catalogs: true
I20260204 08:52:56.620055 2028 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40830:
table { table_name: "default.Table_7bcbc3cfbc41406cba3ed79125a46e82" } modify_external_catalogs: true
I20260204 08:52:56.620925 1138 tablet_service.cc:1552] Processing DeleteTablet for tablet 822cc58f9ac042bf8b21c8ac3fc30ec2 with delete_type TABLET_DATA_DELETED (replaced by tablet f65514e5263e4f86a34195435040910f) from {username='slave'} at 127.0.0.1:56370
I20260204 08:52:56.627067 2117 ts_tablet_manager.cc:1929] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:56.627218 2117 log.cc:1199] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/822cc58f9ac042bf8b21c8ac3fc30ec2
I20260204 08:52:56.627682 2117 ts_tablet_manager.cc:1950] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:52:56.627760 1002 tablet_service.cc:1505] Processing CreateTablet for tablet f65514e5263e4f86a34195435040910f (DEFAULT_TABLE table=default.table_f087faf331654fae99b16adcfb5f705f [id=b3b4145c845b417aa7582e41fb48b564]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:56.628225 1002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f65514e5263e4f86a34195435040910f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:56.628877 2117 tablet_replica.cc:333] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:52:56.629122 2117 raft_consensus.cc:2243] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:56.629333 2117 raft_consensus.cc:2272] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:56.629930 2117 ts_tablet_manager.cc:1916] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:56.630362 1140 tablet_service.cc:1505] Processing CreateTablet for tablet d269b616e72f442e8a47f0c9192db8f8 (DEFAULT_TABLE table=default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:56.630759 1140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d269b616e72f442e8a47f0c9192db8f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:56.614698 2095 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: LEADER
I20260204 08:52:56.631872 1138 tablet_service.cc:1505] Processing CreateTablet for tablet f65514e5263e4f86a34195435040910f (DEFAULT_TABLE table=default.table_f087faf331654fae99b16adcfb5f705f [id=b3b4145c845b417aa7582e41fb48b564]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:56.632261 1138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f65514e5263e4f86a34195435040910f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:56.632369 1275 tablet_service.cc:1552] Processing DeleteTablet for tablet 822cc58f9ac042bf8b21c8ac3fc30ec2 with delete_type TABLET_DATA_DELETED (replaced by tablet f65514e5263e4f86a34195435040910f) from {username='slave'} at 127.0.0.1:45772
I20260204 08:52:56.632965 1276 tablet_service.cc:1552] Processing DeleteTablet for tablet 0771de581fcb4c90823dfaa8bf339cc9 with delete_type TABLET_DATA_DELETED (replaced by tablet eda7388ec12f4a7386823dadedaf2e2f) from {username='slave'} at 127.0.0.1:45772
I20260204 08:52:56.633055 1274 tablet_service.cc:1505] Processing CreateTablet for tablet f65514e5263e4f86a34195435040910f (DEFAULT_TABLE table=default.table_f087faf331654fae99b16adcfb5f705f [id=b3b4145c845b417aa7582e41fb48b564]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:56.633342 1273 tablet_service.cc:1505] Processing CreateTablet for tablet d269b616e72f442e8a47f0c9192db8f8 (DEFAULT_TABLE table=default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:56.633742 1274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f65514e5263e4f86a34195435040910f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:56.636565 2017 catalog_manager.cc:4985] TS 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): tablet 822cc58f9ac042bf8b21c8ac3fc30ec2 (table default.table_f087faf331654fae99b16adcfb5f705f [id=b3b4145c845b417aa7582e41fb48b564]) successfully deleted
I20260204 08:52:56.636695 1273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d269b616e72f442e8a47f0c9192db8f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:56.638587 1001 tablet_service.cc:1505] Processing CreateTablet for tablet eda7388ec12f4a7386823dadedaf2e2f (DEFAULT_TABLE table=default.table_1389d07a897b48439a706dce8abe591b [id=5a415846f1274434994325bb65b31911]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:56.639040 1001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eda7388ec12f4a7386823dadedaf2e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:56.640715 1276 tablet_service.cc:1505] Processing CreateTablet for tablet eda7388ec12f4a7386823dadedaf2e2f (DEFAULT_TABLE table=default.table_1389d07a897b48439a706dce8abe591b [id=5a415846f1274434994325bb65b31911]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:56.642036 1276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eda7388ec12f4a7386823dadedaf2e2f. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f4f5727cf99 kudu::Thread::SuperviseThread() at ??:0
I20260204 08:52:56.648066 1137 tablet_service.cc:1505] Processing CreateTablet for tablet eda7388ec12f4a7386823dadedaf2e2f (DEFAULT_TABLE table=default.table_1389d07a897b48439a706dce8abe591b [id=5a415846f1274434994325bb65b31911]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:56.648605 1137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eda7388ec12f4a7386823dadedaf2e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:56.649763 2119 tablet_replica.cc:333] T 0771de581fcb4c90823dfaa8bf339cc9 P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:52:56.649984 2119 raft_consensus.cc:2243] T 0771de581fcb4c90823dfaa8bf339cc9 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:56.650231 2119 raft_consensus.cc:2272] T 0771de581fcb4c90823dfaa8bf339cc9 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:56.651098 2119 ts_tablet_manager.cc:1916] T 0771de581fcb4c90823dfaa8bf339cc9 P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f4f5a8896db start_thread at ??:0
@ 0x7f4f533b871f clone at ??:0
I20260204 08:52:56.660373 2119 ts_tablet_manager.cc:1929] T 0771de581fcb4c90823dfaa8bf339cc9 P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:56.660523 2119 log.cc:1199] T 0771de581fcb4c90823dfaa8bf339cc9 P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/0771de581fcb4c90823dfaa8bf339cc9
I20260204 08:52:56.660952 2119 ts_tablet_manager.cc:1950] T 0771de581fcb4c90823dfaa8bf339cc9 P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:52:56.661985 2119 tablet_replica.cc:333] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:52:56.662055 2124 tablet_bootstrap.cc:492] T d269b616e72f442e8a47f0c9192db8f8 P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:56.662415 2119 raft_consensus.cc:2243] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:56.662606 2119 raft_consensus.cc:2272] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:56.662783 1000 tablet_service.cc:1505] Processing CreateTablet for tablet d269b616e72f442e8a47f0c9192db8f8 (DEFAULT_TABLE table=default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:56.663147 2119 ts_tablet_manager.cc:1916] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:56.663240 1000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d269b616e72f442e8a47f0c9192db8f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:56.664999 2124 tablet_bootstrap.cc:654] T d269b616e72f442e8a47f0c9192db8f8 P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:56.670303 2119 ts_tablet_manager.cc:1929] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:56.670469 2119 log.cc:1199] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/822cc58f9ac042bf8b21c8ac3fc30ec2
I20260204 08:52:56.670850 2119 ts_tablet_manager.cc:1950] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:52:56.672356 2122 tablet_bootstrap.cc:492] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:56.672952 2123 tablet_bootstrap.cc:492] T d269b616e72f442e8a47f0c9192db8f8 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:56.673964 2122 tablet_bootstrap.cc:654] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:56.675226 2123 tablet_bootstrap.cc:654] T d269b616e72f442e8a47f0c9192db8f8 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:56.676169 2124 tablet_bootstrap.cc:492] T d269b616e72f442e8a47f0c9192db8f8 P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:56.676371 2124 ts_tablet_manager.cc:1403] T d269b616e72f442e8a47f0c9192db8f8 P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260204 08:52:56.677408 2124 raft_consensus.cc:359] T d269b616e72f442e8a47f0c9192db8f8 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:56.677613 2124 raft_consensus.cc:385] T d269b616e72f442e8a47f0c9192db8f8 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:56.677706 2122 tablet_bootstrap.cc:492] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:56.677706 2124 raft_consensus.cc:740] T d269b616e72f442e8a47f0c9192db8f8 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:56.677852 2122 ts_tablet_manager.cc:1403] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260204 08:52:56.677978 2124 consensus_queue.cc:260] T d269b616e72f442e8a47f0c9192db8f8 P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:56.678634 2122 raft_consensus.cc:359] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:56.678828 2117 ts_tablet_manager.cc:1929] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:56.678982 2117 log.cc:1199] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/0771de581fcb4c90823dfaa8bf339cc9
I20260204 08:52:56.678851 2122 raft_consensus.cc:385] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:56.679250 2122 raft_consensus.cc:740] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:56.679407 2117 ts_tablet_manager.cc:1950] T 0771de581fcb4c90823dfaa8bf339cc9 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:52:56.679528 2122 consensus_queue.cc:260] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:56.680253 2122 ts_tablet_manager.cc:1434] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260204 08:52:56.680577 2122 tablet_bootstrap.cc:492] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:56.680557 2124 ts_tablet_manager.cc:1434] T d269b616e72f442e8a47f0c9192db8f8 P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260204 08:52:56.681387 2121 tablet_replica.cc:333] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:52:56.681576 2121 raft_consensus.cc:2243] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:52:56.681762 2124 tablet_bootstrap.cc:492] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:56.681950 2121 raft_consensus.cc:2272] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:56.682305 2122 tablet_bootstrap.cc:654] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:56.682524 2121 ts_tablet_manager.cc:1916] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:56.683488 2124 tablet_bootstrap.cc:654] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:56.692649 2124 tablet_bootstrap.cc:492] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:56.692848 2124 ts_tablet_manager.cc:1403] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260204 08:52:56.693671 2124 raft_consensus.cc:359] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:56.693878 2124 raft_consensus.cc:385] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:56.693965 2124 raft_consensus.cc:740] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:56.694275 2124 consensus_queue.cc:260] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:56.694738 2123 tablet_bootstrap.cc:492] T d269b616e72f442e8a47f0c9192db8f8 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:56.694885 2123 ts_tablet_manager.cc:1403] T d269b616e72f442e8a47f0c9192db8f8 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.000s
I20260204 08:52:56.695678 2123 raft_consensus.cc:359] T d269b616e72f442e8a47f0c9192db8f8 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:56.695865 2123 raft_consensus.cc:385] T d269b616e72f442e8a47f0c9192db8f8 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:56.695948 2123 raft_consensus.cc:740] T d269b616e72f442e8a47f0c9192db8f8 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:56.696012 2124 ts_tablet_manager.cc:1434] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260204 08:52:56.696169 2123 consensus_queue.cc:260] T d269b616e72f442e8a47f0c9192db8f8 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:56.696702 2123 ts_tablet_manager.cc:1434] T d269b616e72f442e8a47f0c9192db8f8 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260204 08:52:56.696993 2123 tablet_bootstrap.cc:492] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:56.697278 2124 tablet_bootstrap.cc:492] T eda7388ec12f4a7386823dadedaf2e2f P 1234acd7d0db405085a5d95795007134: Bootstrap starting.
I20260204 08:52:56.698817 2123 tablet_bootstrap.cc:654] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:56.698853 2124 tablet_bootstrap.cc:654] T eda7388ec12f4a7386823dadedaf2e2f P 1234acd7d0db405085a5d95795007134: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:56.698942 2121 ts_tablet_manager.cc:1929] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:56.699136 2121 log.cc:1199] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/822cc58f9ac042bf8b21c8ac3fc30ec2
I20260204 08:52:56.700759 2124 tablet_bootstrap.cc:492] T eda7388ec12f4a7386823dadedaf2e2f P 1234acd7d0db405085a5d95795007134: No bootstrap required, opened a new log
I20260204 08:52:56.700897 2124 ts_tablet_manager.cc:1403] T eda7388ec12f4a7386823dadedaf2e2f P 1234acd7d0db405085a5d95795007134: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260204 08:52:56.702613 2124 raft_consensus.cc:359] T eda7388ec12f4a7386823dadedaf2e2f P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:56.702827 2124 raft_consensus.cc:385] T eda7388ec12f4a7386823dadedaf2e2f P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:56.702912 2124 raft_consensus.cc:740] T eda7388ec12f4a7386823dadedaf2e2f P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1234acd7d0db405085a5d95795007134, State: Initialized, Role: FOLLOWER
I20260204 08:52:56.703208 2124 consensus_queue.cc:260] T eda7388ec12f4a7386823dadedaf2e2f P 1234acd7d0db405085a5d95795007134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:56.704879 2124 ts_tablet_manager.cc:1434] T eda7388ec12f4a7386823dadedaf2e2f P 1234acd7d0db405085a5d95795007134: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260204 08:52:56.706264 2121 ts_tablet_manager.cc:1950] T 822cc58f9ac042bf8b21c8ac3fc30ec2 P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:52:56.707020 2123 tablet_bootstrap.cc:492] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:56.707330 2121 tablet_replica.cc:333] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:52:56.707333 2123 ts_tablet_manager.cc:1403] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260204 08:52:56.707525 2121 raft_consensus.cc:2243] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:52:56.707837 2121 raft_consensus.cc:2272] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:56.708086 2123 raft_consensus.cc:359] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:56.708268 2123 raft_consensus.cc:385] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:56.708379 2123 raft_consensus.cc:740] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:56.708508 2121 ts_tablet_manager.cc:1916] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:56.708631 2123 consensus_queue.cc:260] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:56.709080 2123 ts_tablet_manager.cc:1434] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260204 08:52:56.709394 2123 tablet_bootstrap.cc:492] T eda7388ec12f4a7386823dadedaf2e2f P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:56.711066 2123 tablet_bootstrap.cc:654] T eda7388ec12f4a7386823dadedaf2e2f P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:56.713251 2123 tablet_bootstrap.cc:492] T eda7388ec12f4a7386823dadedaf2e2f P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:56.713416 2123 ts_tablet_manager.cc:1403] T eda7388ec12f4a7386823dadedaf2e2f P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260204 08:52:56.714159 2123 raft_consensus.cc:359] T eda7388ec12f4a7386823dadedaf2e2f P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:56.714355 2123 raft_consensus.cc:385] T eda7388ec12f4a7386823dadedaf2e2f P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:56.714438 2123 raft_consensus.cc:740] T eda7388ec12f4a7386823dadedaf2e2f P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:56.714509 2121 ts_tablet_manager.cc:1929] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:56.714634 2121 log.cc:1199] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/0771de581fcb4c90823dfaa8bf339cc9
I20260204 08:52:56.714649 2123 consensus_queue.cc:260] T eda7388ec12f4a7386823dadedaf2e2f P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:56.715065 2121 ts_tablet_manager.cc:1950] T 0771de581fcb4c90823dfaa8bf339cc9 P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:52:56.715200 2123 ts_tablet_manager.cc:1434] T eda7388ec12f4a7386823dadedaf2e2f P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260204 08:52:56.718256 2122 tablet_bootstrap.cc:492] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:56.718444 2122 ts_tablet_manager.cc:1403] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.038s user 0.010s sys 0.004s
I20260204 08:52:56.719251 2122 raft_consensus.cc:359] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:56.719453 2122 raft_consensus.cc:385] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:56.719532 2122 raft_consensus.cc:740] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:56.719784 2122 consensus_queue.cc:260] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:56.720350 2122 ts_tablet_manager.cc:1434] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260204 08:52:56.720716 2122 tablet_bootstrap.cc:492] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae: Bootstrap starting.
I20260204 08:52:56.723064 2122 tablet_bootstrap.cc:654] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:56.725358 2122 tablet_bootstrap.cc:492] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae: No bootstrap required, opened a new log
I20260204 08:52:56.725510 2122 ts_tablet_manager.cc:1403] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260204 08:52:56.726332 2122 raft_consensus.cc:359] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:56.726552 2122 raft_consensus.cc:385] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:56.726642 2122 raft_consensus.cc:740] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Initialized, Role: FOLLOWER
I20260204 08:52:56.726904 2122 consensus_queue.cc:260] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:56.727555 2122 ts_tablet_manager.cc:1434] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260204 08:52:56.744113 2092 raft_consensus.cc:493] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:56.744345 2092 raft_consensus.cc:515] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:56.745096 2092 leader_election.cc:290] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:56.745738 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d269b616e72f442e8a47f0c9192db8f8" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:56.745748 1158 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d269b616e72f442e8a47f0c9192db8f8" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:56.745949 1158 raft_consensus.cc:2468] T d269b616e72f442e8a47f0c9192db8f8 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 0.
I20260204 08:52:56.745970 1296 raft_consensus.cc:2468] T d269b616e72f442e8a47f0c9192db8f8 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 0.
I20260204 08:52:56.746500 958 leader_election.cc:304] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:56.746809 2092 raft_consensus.cc:2804] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:56.746927 2092 raft_consensus.cc:493] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:56.747004 2092 raft_consensus.cc:3060] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:56.749775 2092 raft_consensus.cc:515] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:56.752488 2092 leader_election.cc:290] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 election: Requested vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:56.753029 1158 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d269b616e72f442e8a47f0c9192db8f8" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:56.753051 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d269b616e72f442e8a47f0c9192db8f8" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:56.753255 1158 raft_consensus.cc:3060] T d269b616e72f442e8a47f0c9192db8f8 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:56.753274 1296 raft_consensus.cc:3060] T d269b616e72f442e8a47f0c9192db8f8 P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:56.756068 1296 raft_consensus.cc:2468] T d269b616e72f442e8a47f0c9192db8f8 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 1.
I20260204 08:52:56.756095 1158 raft_consensus.cc:2468] T d269b616e72f442e8a47f0c9192db8f8 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 1.
I20260204 08:52:56.756705 958 leader_election.cc:304] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae; no voters:
I20260204 08:52:56.757417 2141 raft_consensus.cc:2804] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:56.757642 2141 raft_consensus.cc:697] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 LEADER]: Becoming Leader. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Running, Role: LEADER
I20260204 08:52:56.758332 2141 consensus_queue.cc:237] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:56.779059 1984 raft_consensus.cc:493] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:56.779246 1984 raft_consensus.cc:515] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:56.779983 1984 leader_election.cc:290] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:56.780598 1019 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f65514e5263e4f86a34195435040910f" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:52:56.780707 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f65514e5263e4f86a34195435040910f" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:56.780814 1019 raft_consensus.cc:2468] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:56.780961 1296 raft_consensus.cc:2468] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 0.
I20260204 08:52:56.781333 1096 leader_election.cc:304] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:56.781611 1984 raft_consensus.cc:2804] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:56.781734 1984 raft_consensus.cc:493] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:56.781826 1984 raft_consensus.cc:3060] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:56.783717 1984 raft_consensus.cc:515] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:56.784363 1984 leader_election.cc:290] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Requested vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:52:56.785143 1019 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f65514e5263e4f86a34195435040910f" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:52:56.785359 1019 raft_consensus.cc:3060] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:56.785463 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f65514e5263e4f86a34195435040910f" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:56.785653 1296 raft_consensus.cc:3060] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:56.788372 1296 raft_consensus.cc:2468] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:56.788555 1019 raft_consensus.cc:2468] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:52:56.789083 1096 leader_election.cc:304] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:56.789897 1984 raft_consensus.cc:2804] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:56.790463 1984 raft_consensus.cc:697] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Becoming Leader. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Running, Role: LEADER
I20260204 08:52:56.790882 1984 consensus_queue.cc:237] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
W20260204 08:52:56.877964 955 connection.cc:537] server connection from 127.0.0.1:38438 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:52:56.877993 1232 connection.cc:537] server connection from 127.0.0.1:45772 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:52:56.878449 957 connection.cc:537] client connection to 127.0.104.189:37005 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:52:56.878485 1353 connection.cc:537] client connection to 127.0.104.189:37005 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:52:56.878727 1231 connection.cc:537] client connection to 127.0.104.189:37005 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:52:56.879211 1353 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.104.189:37005)
W20260204 08:52:56.879545 1353 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.104.189:37005)
W20260204 08:52:56.879743 1353 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.104.189:37005)
W20260204 08:52:56.880028 1095 connection.cc:537] client connection to 127.0.104.189:37005 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:52:56.881103 1093 connection.cc:537] server connection from 127.0.0.1:56370 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:52:56.881229 1353 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.104.189:37005)
I20260204 08:52:56.918466 1846 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:52:56.918649 1846 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:52:57.063023 2141 raft_consensus.cc:493] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:52:57.063246 2141 raft_consensus.cc:515] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:57.063856 2141 leader_election.cc:290] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:57.064441 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eda7388ec12f4a7386823dadedaf2e2f" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134" is_pre_election: true
I20260204 08:52:57.064492 1158 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eda7388ec12f4a7386823dadedaf2e2f" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
I20260204 08:52:57.064671 1296 raft_consensus.cc:2468] T eda7388ec12f4a7386823dadedaf2e2f P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 0.
I20260204 08:52:57.064697 1158 raft_consensus.cc:2468] T eda7388ec12f4a7386823dadedaf2e2f P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 0.
I20260204 08:52:57.065070 958 leader_election.cc:304] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:57.065380 2141 raft_consensus.cc:2804] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260204 08:52:57.065464 2141 raft_consensus.cc:493] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:52:57.065558 2141 raft_consensus.cc:3060] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:57.067495 2141 raft_consensus.cc:515] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:57.068027 2141 leader_election.cc:290] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 election: Requested vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:57.068620 1296 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eda7388ec12f4a7386823dadedaf2e2f" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1234acd7d0db405085a5d95795007134"
I20260204 08:52:57.068627 1158 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eda7388ec12f4a7386823dadedaf2e2f" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070"
I20260204 08:52:57.068804 1296 raft_consensus.cc:3060] T eda7388ec12f4a7386823dadedaf2e2f P 1234acd7d0db405085a5d95795007134 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:57.068820 1158 raft_consensus.cc:3060] T eda7388ec12f4a7386823dadedaf2e2f P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Advancing to term 1
I20260204 08:52:57.070633 1158 raft_consensus.cc:2468] T eda7388ec12f4a7386823dadedaf2e2f P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 1.
I20260204 08:52:57.070648 1296 raft_consensus.cc:2468] T eda7388ec12f4a7386823dadedaf2e2f P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d471268b9e643e5bbbf8bde1644a5ae in term 1.
I20260204 08:52:57.071115 958 leader_election.cc:304] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:52:57.071394 2141 raft_consensus.cc:2804] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader election won for term 1
I20260204 08:52:57.071544 2141 raft_consensus.cc:697] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 LEADER]: Becoming Leader. State: Replica: 6d471268b9e643e5bbbf8bde1644a5ae, State: Running, Role: LEADER
I20260204 08:52:57.071998 2141 consensus_queue.cc:237] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:52:57.132227 2150 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:52:57.132465 2150 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:57.133167 2150 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:52:57.134639 1865 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:52:57.134800 1863 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:52:57.134948 1863 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:52:57.135192 2150 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260204 08:52:57.518128 2141 consensus_queue.cc:1048] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:57.523392 2141 consensus_queue.cc:1048] T d269b616e72f442e8a47f0c9192db8f8 P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:57.773808 2161 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:52:57.773993 2161 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:57.774653 2161 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:52:57.775449 1865 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111) [suppressed 7 similar messages]
W20260204 08:52:57.776566 1863 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:52:57.776711 1865 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:52:57.776855 1865 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:52:57.777096 2161 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260204 08:52:57.892273 2143 consensus_queue.cc:1048] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:57.906805 2143 consensus_queue.cc:1048] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:57.945292 2156 consensus_queue.cc:1048] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:57.952888 2141 consensus_queue.cc:1048] T eda7388ec12f4a7386823dadedaf2e2f P 6d471268b9e643e5bbbf8bde1644a5ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260204 08:52:58.720026 2176 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:52:58.720244 2176 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:58.721263 2176 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:52:58.723361 1863 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:52:58.723547 1865 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:52:58.723690 1865 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:52:58.723946 2176 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260204 08:52:58.875588 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
/tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:37005
--webserver_interface=127.0.104.189
--webserver_port=43227
--builtin_ntp_servers=127.0.104.148:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260204 08:52:59.243007 2180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260204 08:52:59.243373 2180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260204 08:52:59.243494 2180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260204 08:52:59.252421 2180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260204 08:52:59.252537 2180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260204 08:52:59.252602 2180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260204 08:52:59.252664 2180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260204 08:52:59.262976 2180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:37005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.104.189
--webserver_port=43227
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0efbebd487e7d0c7f21ab0d5a91e0fa70046af83
build type FASTDEBUG
built by None at 04 Feb 2026 08:43:20 UTC on 24a791456cd2
build id 10340
ASAN enabled
I20260204 08:52:59.264717 2180 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260204 08:52:59.266844 2180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260204 08:52:59.275266 2188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:59.275275 2189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:52:59.276126 2191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260204 08:52:59.276345 2180 server_base.cc:1060] running on GCE node
I20260204 08:52:59.277197 2180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260204 08:52:59.278743 2180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260204 08:52:59.280025 2180 hybrid_clock.cc:648] HybridClock initialized: now 1770195179279937 us; error 53 us; skew 500 ppm
I20260204 08:52:59.280529 2180 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260204 08:52:59.283149 2180 webserver.cc:492] Webserver started at http://127.0.104.189:43227/ using document root <none> and password file <none>
I20260204 08:52:59.283910 2180 fs_manager.cc:362] Metadata directory not provided
I20260204 08:52:59.284013 2180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260204 08:52:59.290221 2180 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260204 08:52:59.293457 2197 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260204 08:52:59.296988 2180 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.004s sys 0.001s
I20260204 08:52:59.297228 2180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data,/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
uuid: "d083ecc9895345e5b06cd00ef0246c71"
format_stamp: "Formatted at 2026-02-04 08:52:41 on dist-test-slave-vp4h"
I20260204 08:52:59.298102 2180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260204 08:52:59.345403 2180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260204 08:52:59.346312 2180 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260204 08:52:59.346591 2180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260204 08:52:59.370725 2180 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.189:37005
I20260204 08:52:59.370726 2249 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.189:37005 every 8 connection(s)
I20260204 08:52:59.372949 2180 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data/info.pb
I20260204 08:52:59.376706 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu as pid 2180
I20260204 08:52:59.381927 2250 sys_catalog.cc:263] Verifying existing consensus state
I20260204 08:52:59.386101 2250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Bootstrap starting.
I20260204 08:52:59.406296 2250 log.cc:826] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Log is configured to *not* fsync() on all Append() calls
I20260204 08:52:59.447258 1207 heartbeater.cc:344] Connected to a master server at 127.0.104.189:37005
I20260204 08:52:59.465492 1343 heartbeater.cc:344] Connected to a master server at 127.0.104.189:37005
I20260204 08:52:59.488117 2250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Bootstrap replayed 1/1 log segments. Stats: ops{read=96 overwritten=0 applied=96 ignored=0} inserts{seen=65 ignored=0} mutations{seen=82 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260204 08:52:59.488687 2250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Bootstrap complete.
I20260204 08:52:59.493925 2250 raft_consensus.cc:359] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:59.494639 2250 raft_consensus.cc:740] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: d083ecc9895345e5b06cd00ef0246c71, State: Initialized, Role: FOLLOWER
I20260204 08:52:59.495294 2250 consensus_queue.cc:260] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 96, Last appended: 4.96, Last appended by leader: 96, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:59.496279 2258 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:59.496874 2258 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:59.497570 2250 sys_catalog.cc:565] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: configured and running, proceeding with master startup.
I20260204 08:52:59.514443 2272 catalog_manager.cc:1269] Loaded cluster ID: 41c7773c62174f70835c9cfa744e1148
I20260204 08:52:59.514561 2272 catalog_manager.cc:1562] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: loading cluster ID for follower catalog manager: success
I20260204 08:52:59.519742 2272 catalog_manager.cc:1584] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: acquiring CA information for follower catalog manager: success
I20260204 08:52:59.525373 2272 catalog_manager.cc:1612] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260204 08:52:59.554253 2180 master_runner.cc:186] Error getting master registration for 127.0.104.188:38981: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111),
I20260204 08:52:59.559803 2180 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:52:59.568254 2277 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:52:59.568588 2277 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:59.573225 2277 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:52:59.574772 1863 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:52:59.579402 2225 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" candidate_term: 5 candidate_status { last_received { term: 4 index: 96 } } ignore_live_leader: false dest_uuid: "d083ecc9895345e5b06cd00ef0246c71" is_pre_election: true
I20260204 08:52:59.579960 2225 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebeeb5d4823b41b3a8e37546ff2f6e34 in term 4.
I20260204 08:52:59.580569 1865 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d083ecc9895345e5b06cd00ef0246c71, ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69
I20260204 08:52:59.580991 2277 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260204 08:52:59.581758 2277 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:52:59.581911 2277 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 4 FOLLOWER]: Advancing to term 5
I20260204 08:52:59.584486 2277 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:59.585180 2277 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 election: Requested vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:52:59.586956 1863 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:52:59.588840 2225 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" candidate_term: 5 candidate_status { last_received { term: 4 index: 96 } } ignore_live_leader: false dest_uuid: "d083ecc9895345e5b06cd00ef0246c71"
I20260204 08:52:59.589054 2225 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 4 FOLLOWER]: Advancing to term 5
I20260204 08:52:59.593639 2225 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebeeb5d4823b41b3a8e37546ff2f6e34 in term 5.
I20260204 08:52:59.594218 1865 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d083ecc9895345e5b06cd00ef0246c71, ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69
I20260204 08:52:59.594573 2277 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 5 FOLLOWER]: Leader election won for term 5
I20260204 08:52:59.595055 2277 raft_consensus.cc:697] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 5 LEADER]: Becoming Leader. State: Replica: ebeeb5d4823b41b3a8e37546ff2f6e34, State: Running, Role: LEADER
I20260204 08:52:59.595535 2277 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 96, Committed index: 96, Last appended: 4.96, Last appended by leader: 96, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:52:59.597419 2281 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebeeb5d4823b41b3a8e37546ff2f6e34. Latest consensus state: current_term: 5 leader_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:59.597734 2281 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: LEADER
I20260204 08:52:59.598479 2283 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260204 08:52:59.601647 2283 catalog_manager.cc:679] Loaded metadata for table default.table_f388b768c4b14290a90dd29962cc3fc4 [id=105a9508f80c4190a40ab4ab66af2872]
I20260204 08:52:59.602174 2283 catalog_manager.cc:679] Loaded metadata for table default.table_a638f2d4f2654854afdb0b065ee83f48 [id=12e17f7b128b4db99ae9041dba0602b5]
I20260204 08:52:59.602557 2283 catalog_manager.cc:679] Loaded metadata for table default.table_7bcbc3cfbc41406cba3ed79125a46e82 [id=180667e663144b7ea79638946d6c76cc]
I20260204 08:52:59.602900 2283 catalog_manager.cc:679] Loaded metadata for table default.table_e7fe420574dc40cdb70c4602f5a6e0db [id=19930831c99540688fb9a1e6ebfd8f82]
I20260204 08:52:59.603266 2283 catalog_manager.cc:679] Loaded metadata for table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]
I20260204 08:52:59.603686 2283 catalog_manager.cc:679] Loaded metadata for table default.table_97ec63740f2b4928af12e35b5eaa29cd [id=2a58ed39876b46aba637d263234e55d9]
I20260204 08:52:59.604020 2283 catalog_manager.cc:679] Loaded metadata for table default.table_f8273ef1a407421085918a8e3827f5f6 [id=2df47e721cd642e4b6fd5e61e172749c]
I20260204 08:52:59.604569 2283 catalog_manager.cc:679] Loaded metadata for table default.table_73e876d60c444a5daa316e4bf7158614 [id=42a606fb50bf4c58be67f67afab90d53]
I20260204 08:52:59.604894 2283 catalog_manager.cc:679] Loaded metadata for table default.table_1389d07a897b48439a706dce8abe591b [id=5a415846f1274434994325bb65b31911]
I20260204 08:52:59.605234 2283 catalog_manager.cc:679] Loaded metadata for table default.table_c97013dd16f84d8ba7cd528295f5ad49 [id=5f9f5b0850774ff9b8ecb93e141ff768]
I20260204 08:52:59.605549 2283 catalog_manager.cc:679] Loaded metadata for table default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807]
I20260204 08:52:59.605878 2283 catalog_manager.cc:679] Loaded metadata for table default.table_b4244312363c4346b110598eb01428a3 [id=7d4edb505f7940668b492a9d592a3e61]
I20260204 08:52:59.606222 2283 catalog_manager.cc:679] Loaded metadata for table default.table_9f93a995778f4fe6a4667cff0ef494b9 [id=893f125488f745a5b0bc9ec3d07da00b]
I20260204 08:52:59.606604 2283 catalog_manager.cc:679] Loaded metadata for table default.table_4f900fd574c849aba171e6a64bc43dbd [id=9427e77d6a3842a7a2665d99c8ebb99e]
I20260204 08:52:59.606989 2283 catalog_manager.cc:679] Loaded metadata for table default.table_477defa30c2c40e99360b505ef988a80 [id=b1e79546138b4dcb9c5a68977af78bf0]
I20260204 08:52:59.607342 2283 catalog_manager.cc:679] Loaded metadata for table default.table_f087faf331654fae99b16adcfb5f705f [id=b3b4145c845b417aa7582e41fb48b564]
I20260204 08:52:59.607677 2283 catalog_manager.cc:679] Loaded metadata for table default.table_9e950ede78084ef18f4e1051d9a42a90 [id=c626dc1c029e4b59902b4493f4e16a78]
I20260204 08:52:59.608042 2283 catalog_manager.cc:679] Loaded metadata for table default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294]
I20260204 08:52:59.608500 2283 catalog_manager.cc:679] Loaded metadata for table default.table_780ed695e5324a469614b35852a643cf [id=f29a6ab746244116a4815a632194b3ec]
I20260204 08:52:59.610709 2283 tablet_loader.cc:96] loaded metadata for tablet 043cdafd4b11463bae674695e7a74854 (table default.table_9f93a995778f4fe6a4667cff0ef494b9 [id=893f125488f745a5b0bc9ec3d07da00b])
I20260204 08:52:59.611515 2283 tablet_loader.cc:96] loaded metadata for tablet 2de648ce42af4a2fbd494c8cc3e38bb4 (table default.table_f388b768c4b14290a90dd29962cc3fc4 [id=105a9508f80c4190a40ab4ab66af2872])
I20260204 08:52:59.611842 2283 tablet_loader.cc:96] loaded metadata for tablet 350b048c73bd4208a7aa70b3521869e1 (table default.table_f8273ef1a407421085918a8e3827f5f6 [id=2df47e721cd642e4b6fd5e61e172749c])
I20260204 08:52:59.612150 2283 tablet_loader.cc:96] loaded metadata for tablet 35b597aac0264d36a44ae37c6ffb945e (table default.table_7bcbc3cfbc41406cba3ed79125a46e82 [id=180667e663144b7ea79638946d6c76cc])
I20260204 08:52:59.612658 2283 tablet_loader.cc:96] loaded metadata for tablet 4c26152dcc2147d6b0cb8d1242a7a73b (table default.table_477defa30c2c40e99360b505ef988a80 [id=b1e79546138b4dcb9c5a68977af78bf0])
I20260204 08:52:59.613001 2283 tablet_loader.cc:96] loaded metadata for tablet 5135d9b498314d9098339db68f64207d (table default.table_780ed695e5324a469614b35852a643cf [id=f29a6ab746244116a4815a632194b3ec])
I20260204 08:52:59.613334 2283 tablet_loader.cc:96] loaded metadata for tablet 52abcf65cd4840fdb7719ada0979c7a1 (table default.table_a638f2d4f2654854afdb0b065ee83f48 [id=12e17f7b128b4db99ae9041dba0602b5])
I20260204 08:52:59.613797 2283 tablet_loader.cc:96] loaded metadata for tablet 5fe7a46289d84efcb7b1455bd317d279 (table default.table_b4244312363c4346b110598eb01428a3 [id=7d4edb505f7940668b492a9d592a3e61])
I20260204 08:52:59.614550 2283 tablet_loader.cc:96] loaded metadata for tablet 80f72b647ee643afabcc5cafa9c43b13 (table default.table_9e950ede78084ef18f4e1051d9a42a90 [id=c626dc1c029e4b59902b4493f4e16a78])
I20260204 08:52:59.614878 2283 tablet_loader.cc:96] loaded metadata for tablet 81f37ac4f50a440d8e7292bfa05a0a7d (table default.table_73e876d60c444a5daa316e4bf7158614 [id=42a606fb50bf4c58be67f67afab90d53])
I20260204 08:52:59.616091 2283 tablet_loader.cc:96] loaded metadata for tablet c9ce4ab8ce1f48868b890df81e551eed (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421])
I20260204 08:52:59.616465 2283 tablet_loader.cc:96] loaded metadata for tablet cb72d1554d9b4c33b6cd7c01fad765db (table default.table_97ec63740f2b4928af12e35b5eaa29cd [id=2a58ed39876b46aba637d263234e55d9])
I20260204 08:52:59.616922 2283 tablet_loader.cc:96] loaded metadata for tablet d269b616e72f442e8a47f0c9192db8f8 (table default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807])
I20260204 08:52:59.617484 2283 tablet_loader.cc:96] loaded metadata for tablet deef771e18ac44a5b81f258fe8e6f3c4 (table default.table_4f900fd574c849aba171e6a64bc43dbd [id=9427e77d6a3842a7a2665d99c8ebb99e])
I20260204 08:52:59.617862 2283 tablet_loader.cc:96] loaded metadata for tablet e31d044c9bb14ceb8d637977966630b9 (table default.table_c97013dd16f84d8ba7cd528295f5ad49 [id=5f9f5b0850774ff9b8ecb93e141ff768])
I20260204 08:52:59.618185 2283 tablet_loader.cc:96] loaded metadata for tablet ebed8bb53e7947e08d0c960b13bacd8d (table default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294])
I20260204 08:52:59.618489 2283 tablet_loader.cc:96] loaded metadata for tablet eda7388ec12f4a7386823dadedaf2e2f (table default.table_1389d07a897b48439a706dce8abe591b [id=5a415846f1274434994325bb65b31911])
I20260204 08:52:59.618798 2283 tablet_loader.cc:96] loaded metadata for tablet f65514e5263e4f86a34195435040910f (table default.table_f087faf331654fae99b16adcfb5f705f [id=b3b4145c845b417aa7582e41fb48b564])
I20260204 08:52:59.619094 2283 tablet_loader.cc:96] loaded metadata for tablet fe0b2df895c643e7b69fb30a2b0b8e5c (table default.table_e7fe420574dc40cdb70c4602f5a6e0db [id=19930831c99540688fb9a1e6ebfd8f82])
I20260204 08:52:59.619357 2283 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260204 08:52:59.620795 2283 catalog_manager.cc:1269] Loaded cluster ID: 41c7773c62174f70835c9cfa744e1148
I20260204 08:52:59.620887 2283 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260204 08:52:59.622587 2283 catalog_manager.cc:1514] Loading token signing keys...
I20260204 08:52:59.623875 2283 catalog_manager.cc:6038] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Loaded TSK: 0
I20260204 08:52:59.624884 2283 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260204 08:52:59.626194 2283 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260204 08:52:59.627496 2283 catalog_manager.cc:5876] Last processed Hive Metastore notification event ID: 20
I20260204 08:52:59.665027 1877 master_service.cc:954] ReplaceTablet: received request to replace tablet 5fe7a46289d84efcb7b1455bd317d279 from {username='slave'} at 127.0.0.1:34138
I20260204 08:52:59.665570 1879 master_service.cc:954] ReplaceTablet: received request to replace tablet f65514e5263e4f86a34195435040910f from {username='slave'} at 127.0.0.1:34138
I20260204 08:52:59.667026 1870 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34138:
table { table_name: "default.Table_e7fe420574dc40cdb70c4602f5a6e0db" } new_table_name: "default.Table_2cf92b501b134cc2a2c8be92b1530908" modify_external_catalogs: true
I20260204 08:52:59.667732 1871 master_service.cc:954] ReplaceTablet: received request to replace tablet d3c5509788ad4f8c858db788681b0920 from {username='slave'} at 127.0.0.1:34138
I20260204 08:52:59.668332 1878 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34138:
table { table_name: "default.Table_f388b768c4b14290a90dd29962cc3fc4" } modify_external_catalogs: true
I20260204 08:52:59.668612 1878 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34138:
table { table_name: "default.Table_f388b768c4b14290a90dd29962cc3fc4" } modify_external_catalogs: true
I20260204 08:52:59.668942 1871 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34138:
name: "default.Table_c2dfe1ff212d4d2b8d46f11cb24ac67e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:59.669485 1875 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34138:
table { table_name: "default.Table_67b2871860f446a1a5551b16cbbcec4d" } new_table_name: "default.Table_52f9941ea7774cceafc55a0edbdb526c" modify_external_catalogs: true
I20260204 08:52:59.669975 1874 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34138:
table { table_name: "default.Table_9f93a995778f4fe6a4667cff0ef494b9" } new_table_name: "default.Table_b4e59736cf4b48b4b920d4ad64d8c522" modify_external_catalogs: true
I20260204 08:52:59.669037 1876 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34138:
table { table_name: "default.Table_c97013dd16f84d8ba7cd528295f5ad49" } new_table_name: "default.Table_b3a5ec0b6897403bb998ef249c195d5f" modify_external_catalogs: true
I20260204 08:52:59.671481 1873 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34138:
table { table_name: "default.Table_732f1b588b344e49b2f77e17e6551853" } new_table_name: "default.Table_ae16851d63e6436692ececda3e21408e" modify_external_catalogs: true
I20260204 08:52:59.671993 1872 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34138:
table { table_name: "default.Table_b4244312363c4346b110598eb01428a3" } modify_external_catalogs: true
I20260204 08:52:59.672216 1872 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34138:
table { table_name: "default.Table_b4244312363c4346b110598eb01428a3" } modify_external_catalogs: true
I20260204 08:52:59.675686 2225 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 5 FOLLOWER]: Refusing update from remote peer ebeeb5d4823b41b3a8e37546ff2f6e34: Log matching property violated. Preceding OpId in replica: term: 4 index: 96. Preceding OpId from leader: term: 5 index: 98. (index mismatch)
I20260204 08:52:59.676699 2277 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 97, Last known committed idx: 96, Time since last communication: 0.000s
W20260204 08:52:59.677039 1863 consensus_peers.cc:597] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 -> Peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Couldn't send request to peer c1e719e457e94a5da71929720411ae69. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
F20260204 08:52:59.680946 1935 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 23, previous 23; dumping first 4 out of 15 received events: NotificationEvent(eventId=21, eventTime=1770195168, eventType=DROP_TABLE, dbName=default, tableName=table_85f07b4907b24f1ca63791592455b59c, message={"server":"","servicePrincipal":"","db":"default","table":"table_85f07b4907b24f1ca63791592455b59c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_85f07b4907b24f1ca63791592455b59c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195167},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_85f07b4907b24f1ca63791592455b59c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_85f07b4907b24f1ca63791592455b59c\",\"kudu.table_id\":\"158d366d5ccf4b78b73903776a30cb35\",\"transient_lastDdlTime\":\"1770195167\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=22, eventTime=1770195168, eventType=CREATE_TABLE, dbName=default, tableName=table_97ec63740f2b4928af12e35b5eaa29cd, message={"server":"","servicePrincipal":"","db":"default","table":"table_97ec63740f2b4928af12e35b5eaa29cd","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_97ec63740f2b4928af12e35b5eaa29cd\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195168},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_97ec63740f2b4928af12e35b5eaa29cd\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_97ec63740f2b4928af12e35b5eaa29cd\",\"kudu.table_id\":\"2a58ed39876b46aba637d263234e55d9\",\"transient_lastDdlTime\":\"1770195168\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=23, eventTime=1770195168, eventType=DROP_TABLE, dbName=default, tableName=table_279632cf491c4198ad9b91a114c3fc33, message={"server":"","servicePrincipal":"","db":"default","table":"table_279632cf491c4198ad9b91a114c3fc33","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_279632cf491c4198ad9b91a114c3fc33\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195165},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_279632cf491c4198ad9b91a114c3fc33\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_279632cf491c4198ad9b91a114c3fc33\",\"kudu.table_id\":\"da8943eba12e426ba3bac81a64141294\",\"transient_lastDdlTime\":\"1770195165\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=23, eventTime=1770195168, eventType=CREATE_TABLE, dbName=default, tableName=table_477defa30c2c40e99360b505ef988a80, message={"server":"","servicePrincipal":"","db":"default","table":"table_477defa30c2c40e99360b505ef988a80","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_477defa30c2c40e99360b505ef988a80\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195168},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_477defa30c2c40e99360b505ef988a80\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_477defa30c2c40e99360b505ef988a80\",\"kudu.table_id\":\"b1e79546138b4dcb9c5a68977af78bf0\",\"transient_lastDdlTime\":\"1770195168\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f5dbaab0dcd google::LogMessage::Fail() at ??:0
@ 0x7f5dbaab4b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f5dbaab07cc google::LogMessage::Flush() at ??:0
@ 0x7f5dbaab0a11 google::LogMessage::~LogMessage() at ??:0
I20260204 08:52:59.697201 2258 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebeeb5d4823b41b3a8e37546ff2f6e34. Latest consensus state: current_term: 5 leader_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:59.697484 2258 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:52:59.698597 2284 mvcc.cc:204] Tried to move back new op lower bound from 7250719455941156864 to 7250719455627292672. Current Snapshot: MvccSnapshot[applied={T|T < 7250719455941156864}]
@ 0x7f5dd14e47f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260204 08:52:59.704906 1879 catalog_manager.cc:5941] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Sending DeleteTablet for 3 replicas of tablet f65514e5263e4f86a34195435040910f
I20260204 08:52:59.707054 1879 catalog_manager.cc:6724] ReplaceTablet: tablet f65514e5263e4f86a34195435040910f deleted and replaced by tablet 742c8a8c99464661873f292406ac5027
I20260204 08:52:59.708439 1877 catalog_manager.cc:5941] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Sending DeleteTablet for 3 replicas of tablet 5fe7a46289d84efcb7b1455bd317d279
I20260204 08:52:59.708879 1877 catalog_manager.cc:6724] ReplaceTablet: tablet 5fe7a46289d84efcb7b1455bd317d279 deleted and replaced by tablet abef7b1f7bb6415186e9d5dc9b438604
I20260204 08:52:59.714406 2285 mvcc.cc:204] Tried to move back new op lower bound from 7250719455941156864 to 7250719455627292672. Current Snapshot: MvccSnapshot[applied={T|T < 7250719455941156864}]
@ 0x7f5dd14e30a1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260204 08:52:59.742509 1879 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34138:
name: "default.Table_85f07b4907b24f1ca63791592455b59c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:59.747396 1877 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34138:
name: "default.Table_27cce8dee4984465be213744abfadc9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:52:59.749171 2281 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:59.749385 2281 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: LEADER
@ 0x7f5dd14ead1c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260204 08:52:59.752158 1275 tablet_service.cc:1552] Processing DeleteTablet for tablet f65514e5263e4f86a34195435040910f with delete_type TABLET_DATA_DELETED (replaced by tablet 742c8a8c99464661873f292406ac5027) from {username='slave'} at 127.0.0.1:45786
@ 0x7f5dd14eab99 std::_Function_handler<>::_M_invoke() at ??:0
I20260204 08:52:59.756731 2258 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:52:59.757014 2258 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: FOLLOWER
@ 0xa8970d std::function<>::operator()() at ??:?
I20260204 08:52:59.761977 1140 tablet_service.cc:1552] Processing DeleteTablet for tablet f65514e5263e4f86a34195435040910f with delete_type TABLET_DATA_DELETED (replaced by tablet 742c8a8c99464661873f292406ac5027) from {username='slave'} at 127.0.0.1:56384
I20260204 08:52:59.762369 1138 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fe7a46289d84efcb7b1455bd317d279 with delete_type TABLET_DATA_DELETED (replaced by tablet abef7b1f7bb6415186e9d5dc9b438604) from {username='slave'} at 127.0.0.1:56384
@ 0x7f5dbd825f99 kudu::Thread::SuperviseThread() at ??:0
I20260204 08:52:59.765329 1273 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fe7a46289d84efcb7b1455bd317d279 with delete_type TABLET_DATA_DELETED (replaced by tablet abef7b1f7bb6415186e9d5dc9b438604) from {username='slave'} at 127.0.0.1:45786
I20260204 08:52:59.769419 2299 tablet_replica.cc:333] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:52:59.769665 2299 raft_consensus.cc:2243] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:59.769891 2299 raft_consensus.cc:2272] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:59.770491 2299 ts_tablet_manager.cc:1916] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:59.772383 1000 tablet_service.cc:1552] Processing DeleteTablet for tablet f65514e5263e4f86a34195435040910f with delete_type TABLET_DATA_DELETED (replaced by tablet 742c8a8c99464661873f292406ac5027) from {username='slave'} at 127.0.0.1:38452
I20260204 08:52:59.772506 1002 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fe7a46289d84efcb7b1455bd317d279 with delete_type TABLET_DATA_DELETED (replaced by tablet abef7b1f7bb6415186e9d5dc9b438604) from {username='slave'} at 127.0.0.1:38452
I20260204 08:52:59.775761 2300 tablet_replica.cc:333] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:52:59.775975 2300 raft_consensus.cc:2243] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:52:59.776399 2300 raft_consensus.cc:2272] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:59.776991 2300 ts_tablet_manager.cc:1916] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:59.777659 2299 ts_tablet_manager.cc:1929] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:59.777799 2299 log.cc:1199] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/f65514e5263e4f86a34195435040910f
I20260204 08:52:59.778266 2299 ts_tablet_manager.cc:1950] T f65514e5263e4f86a34195435040910f P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:52:59.779429 2299 tablet_replica.cc:333] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:52:59.779660 2299 raft_consensus.cc:2243] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:59.779829 2299 raft_consensus.cc:2272] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
@ 0x7f5dc0e326db start_thread at ??:0
I20260204 08:52:59.780516 1866 catalog_manager.cc:4985] TS 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): tablet f65514e5263e4f86a34195435040910f (table default.table_f087faf331654fae99b16adcfb5f705f [id=b3b4145c845b417aa7582e41fb48b564]) successfully deleted
@ 0x7f5db996171f clone at ??:0
I20260204 08:52:59.782222 1138 tablet_service.cc:1505] Processing CreateTablet for tablet 742c8a8c99464661873f292406ac5027 (DEFAULT_TABLE table=default.table_f087faf331654fae99b16adcfb5f705f [id=b3b4145c845b417aa7582e41fb48b564]), partition=RANGE (key) PARTITION UNBOUNDED
I20260204 08:52:59.782598 2299 ts_tablet_manager.cc:1916] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:59.782650 1138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 742c8a8c99464661873f292406ac5027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260204 08:52:59.785326 2300 ts_tablet_manager.cc:1929] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:59.785449 2300 log.cc:1199] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/f65514e5263e4f86a34195435040910f
I20260204 08:52:59.785884 2300 ts_tablet_manager.cc:1950] T f65514e5263e4f86a34195435040910f P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:52:59.786914 2300 tablet_replica.cc:333] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:52:59.787149 2300 raft_consensus.cc:2243] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:52:59.787478 2300 raft_consensus.cc:2272] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:59.788025 2300 ts_tablet_manager.cc:1916] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:59.789840 2299 ts_tablet_manager.cc:1929] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:59.789988 2299 log.cc:1199] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/5fe7a46289d84efcb7b1455bd317d279
I20260204 08:52:59.790377 2299 ts_tablet_manager.cc:1950] T 5fe7a46289d84efcb7b1455bd317d279 P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:52:59.794044 2300 ts_tablet_manager.cc:1929] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:59.794234 2300 log.cc:1199] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/5fe7a46289d84efcb7b1455bd317d279
I20260204 08:52:59.794540 2301 tablet_replica.cc:333] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:52:59.794651 2300 ts_tablet_manager.cc:1950] T 5fe7a46289d84efcb7b1455bd317d279 P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:52:59.794754 2301 raft_consensus.cc:2243] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:59.794970 2301 raft_consensus.cc:2272] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:59.795586 2301 ts_tablet_manager.cc:1916] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:59.796248 2303 tablet_bootstrap.cc:492] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070: Bootstrap starting.
I20260204 08:52:59.798163 2303 tablet_bootstrap.cc:654] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070: Neither blocks nor log segments found. Creating new log.
I20260204 08:52:59.814844 2301 ts_tablet_manager.cc:1929] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:59.814977 2301 log.cc:1199] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/f65514e5263e4f86a34195435040910f
I20260204 08:52:59.815398 2301 ts_tablet_manager.cc:1950] T f65514e5263e4f86a34195435040910f P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:52:59.816367 2301 tablet_replica.cc:333] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:52:59.816581 2301 raft_consensus.cc:2243] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:52:59.816766 2301 raft_consensus.cc:2272] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:52:59.817293 2301 ts_tablet_manager.cc:1916] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:52:59.824764 2301 ts_tablet_manager.cc:1929] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:52:59.824949 2301 log.cc:1199] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/5fe7a46289d84efcb7b1455bd317d279
I20260204 08:52:59.825476 2301 ts_tablet_manager.cc:1950] T 5fe7a46289d84efcb7b1455bd317d279 P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:52:59.828115 2303 tablet_bootstrap.cc:492] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070: No bootstrap required, opened a new log
I20260204 08:52:59.828289 2303 ts_tablet_manager.cc:1403] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070: Time spent bootstrapping tablet: real 0.032s user 0.022s sys 0.009s
I20260204 08:52:59.828405 1069 heartbeater.cc:344] Connected to a master server at 127.0.104.189:37005
I20260204 08:52:59.829183 2303 raft_consensus.cc:359] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:59.829396 2303 raft_consensus.cc:385] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260204 08:52:59.829483 2303 raft_consensus.cc:740] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Initialized, Role: FOLLOWER
I20260204 08:52:59.829720 2303 consensus_queue.cc:260] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:52:59.830286 2303 ts_tablet_manager.cc:1434] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260204 08:52:59.831620 2214 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" instance_seqno: 1770195163347042) as {username='slave'} at 127.0.104.129:48809; Asking this server to re-register.
I20260204 08:52:59.833061 1069 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:59.833132 2214 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" instance_seqno: 1770195163874581) as {username='slave'} at 127.0.104.130:42791; Asking this server to re-register.
I20260204 08:52:59.833885 1207 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:59.835302 2214 ts_manager.cc:194] Registered new tserver with Master: 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:52:59.837193 2214 ts_manager.cc:194] Registered new tserver with Master: fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:52:59.970535 2214 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1234acd7d0db405085a5d95795007134" instance_seqno: 1770195164422899) as {username='slave'} at 127.0.104.131:38865; Asking this server to re-register.
I20260204 08:52:59.971405 1343 heartbeater.cc:461] Registering TS with master...
I20260204 08:52:59.973518 2214 ts_manager.cc:194] Registered new tserver with Master: 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:53:00.008224 1229 connection.cc:537] server connection from 127.0.0.1:45786 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:53:00.008251 1093 connection.cc:537] server connection from 127.0.0.1:56384 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:53:00.008312 1230 connection.cc:537] client connection to 127.0.104.190:42471 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:53:00.008710 958 connection.cc:537] server connection from 127.0.0.1:38452 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:53:00.008993 1094 connection.cc:537] client connection to 127.0.104.190:42471 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:53:00.009008 956 connection.cc:537] client connection to 127.0.104.190:42471 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:53:00.009902 1352 connection.cc:537] client connection to 127.0.104.190:42471 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:53:00.010898 1352 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.104.190:42471)
W20260204 08:53:00.011497 1352 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.104.190:42471)
W20260204 08:53:00.011777 1352 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.104.190:42471)
W20260204 08:53:00.012392 1352 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.104.190:42471)
I20260204 08:53:00.265956 2315 raft_consensus.cc:493] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ebeeb5d4823b41b3a8e37546ff2f6e34)
I20260204 08:53:00.266199 2315 raft_consensus.cc:515] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:00.268074 2315 leader_election.cc:290] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:00.269091 2200 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.190:42471: connect: Connection refused (error 111)
W20260204 08:53:00.270265 2199 leader_election.cc:336] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:53:00.270594 2200 leader_election.cc:336] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471): Network error: Client connection negotiation failed: client connection to 127.0.104.190:42471: connect: Connection refused (error 111)
I20260204 08:53:00.270725 2200 leader_election.cc:304] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d083ecc9895345e5b06cd00ef0246c71; no voters: c1e719e457e94a5da71929720411ae69, ebeeb5d4823b41b3a8e37546ff2f6e34
I20260204 08:53:00.271076 2315 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260204 08:53:00.641314 2182 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:00.641536 2182 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:53:00.642518 2182 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:53:00.643344 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:53:00.643572 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:53:00.643699 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:53:00.644013 2182 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:53:00.824954 2320 raft_consensus.cc:493] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ebeeb5d4823b41b3a8e37546ff2f6e34)
I20260204 08:53:00.825160 2320 raft_consensus.cc:515] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:00.825830 2320 leader_election.cc:290] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:00.828174 2200 leader_election.cc:336] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471): Network error: Client connection negotiation failed: client connection to 127.0.104.190:42471: connect: Connection refused (error 111)
W20260204 08:53:00.828404 2199 leader_election.cc:336] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:00.828555 2199 leader_election.cc:304] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d083ecc9895345e5b06cd00ef0246c71; no voters: c1e719e457e94a5da71929720411ae69, ebeeb5d4823b41b3a8e37546ff2f6e34
I20260204 08:53:00.828850 2320 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260204 08:53:01.648643 2324 raft_consensus.cc:493] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ebeeb5d4823b41b3a8e37546ff2f6e34)
I20260204 08:53:01.648850 2324 raft_consensus.cc:515] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:01.649659 2324 leader_election.cc:290] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:01.651985 2199 leader_election.cc:336] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:53:01.652287 2200 leader_election.cc:336] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471): Network error: Client connection negotiation failed: client connection to 127.0.104.190:42471: connect: Connection refused (error 111)
I20260204 08:53:01.652504 2200 leader_election.cc:304] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d083ecc9895345e5b06cd00ef0246c71; no voters: c1e719e457e94a5da71929720411ae69, ebeeb5d4823b41b3a8e37546ff2f6e34
I20260204 08:53:01.652904 2324 raft_consensus.cc:2749] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260204 08:53:02.633188 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
/tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:42471
--webserver_interface=127.0.104.190
--webserver_port=40787
--builtin_ntp_servers=127.0.104.148:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260204 08:53:02.979728 2328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260204 08:53:02.980190 2328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260204 08:53:02.980332 2328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260204 08:53:02.989212 2328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260204 08:53:02.989347 2328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260204 08:53:02.989477 2328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260204 08:53:02.989532 2328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260204 08:53:03.000062 2328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:42471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.104.190
--webserver_port=40787
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0efbebd487e7d0c7f21ab0d5a91e0fa70046af83
build type FASTDEBUG
built by None at 04 Feb 2026 08:43:20 UTC on 24a791456cd2
build id 10340
ASAN enabled
I20260204 08:53:03.001822 2328 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260204 08:53:03.003782 2328 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260204 08:53:03.013095 2335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:53:03.013859 2334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:53:03.014225 2337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260204 08:53:03.015476 2328 server_base.cc:1060] running on GCE node
I20260204 08:53:03.016585 2328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260204 08:53:03.018483 2328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260204 08:53:03.019739 2328 hybrid_clock.cc:648] HybridClock initialized: now 1770195183019677 us; error 52 us; skew 500 ppm
I20260204 08:53:03.020144 2328 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260204 08:53:03.022799 2328 webserver.cc:492] Webserver started at http://127.0.104.190:40787/ using document root <none> and password file <none>
I20260204 08:53:03.023613 2328 fs_manager.cc:362] Metadata directory not provided
I20260204 08:53:03.023734 2328 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260204 08:53:03.030107 2328 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260204 08:53:03.033499 2343 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260204 08:53:03.035089 2328 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260204 08:53:03.035275 2328 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data,/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34"
format_stamp: "Formatted at 2026-02-04 08:52:41 on dist-test-slave-vp4h"
I20260204 08:53:03.036115 2328 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260204 08:53:03.063642 2328 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260204 08:53:03.064709 2328 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260204 08:53:03.065037 2328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260204 08:53:03.085099 2328 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.190:42471
I20260204 08:53:03.085130 2394 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.190:42471 every 8 connection(s)
I20260204 08:53:03.086916 2328 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
I20260204 08:53:03.092279 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu as pid 2328
I20260204 08:53:03.097504 2395 sys_catalog.cc:263] Verifying existing consensus state
I20260204 08:53:03.103809 2395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Bootstrap starting.
I20260204 08:53:03.123131 2395 log.cc:826] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Log is configured to *not* fsync() on all Append() calls
I20260204 08:53:03.200028 2395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Bootstrap replayed 1/1 log segments. Stats: ops{read=100 overwritten=0 applied=100 ignored=0} inserts{seen=67 ignored=0} mutations{seen=86 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260204 08:53:03.200596 2395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Bootstrap complete.
I20260204 08:53:03.206003 2395 raft_consensus.cc:359] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:03.206699 2395 raft_consensus.cc:740] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebeeb5d4823b41b3a8e37546ff2f6e34, State: Initialized, Role: FOLLOWER
I20260204 08:53:03.207350 2395 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 100, Last appended: 5.100, Last appended by leader: 100, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:03.208232 2400 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:53:03.208537 2400 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:53:03.209208 2395 sys_catalog.cc:565] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: configured and running, proceeding with master startup.
I20260204 08:53:03.225713 2414 catalog_manager.cc:1269] Loaded cluster ID: 41c7773c62174f70835c9cfa744e1148
I20260204 08:53:03.225852 2414 catalog_manager.cc:1562] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: loading cluster ID for follower catalog manager: success
I20260204 08:53:03.231523 2414 catalog_manager.cc:1584] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: acquiring CA information for follower catalog manager: success
I20260204 08:53:03.236096 2414 catalog_manager.cc:1612] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260204 08:53:03.244637 2419 raft_consensus.cc:493] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ebeeb5d4823b41b3a8e37546ff2f6e34)
I20260204 08:53:03.244861 2419 raft_consensus.cc:515] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:03.254351 2419 leader_election.cc:290] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
I20260204 08:53:03.251933 2328 master_runner.cc:186] Error getting master registration for 127.0.104.188:38981: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111),
I20260204 08:53:03.255941 2370 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d083ecc9895345e5b06cd00ef0246c71" candidate_term: 6 candidate_status { last_received { term: 5 index: 100 } } ignore_live_leader: false dest_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" is_pre_election: true
I20260204 08:53:03.257592 2328 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:53:03.257848 2370 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d083ecc9895345e5b06cd00ef0246c71 in term 5.
W20260204 08:53:03.258245 2199 leader_election.cc:336] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:03.258441 2199 leader_election.cc:304] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d083ecc9895345e5b06cd00ef0246c71, ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69
I20260204 08:53:03.258745 2419 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260204 08:53:03.258865 2419 raft_consensus.cc:493] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ebeeb5d4823b41b3a8e37546ff2f6e34)
I20260204 08:53:03.258955 2419 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 5 FOLLOWER]: Advancing to term 6
I20260204 08:53:03.260936 2419 raft_consensus.cc:515] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:03.261552 2419 leader_election.cc:290] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 election: Requested vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
I20260204 08:53:03.262074 2370 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d083ecc9895345e5b06cd00ef0246c71" candidate_term: 6 candidate_status { last_received { term: 5 index: 100 } } ignore_live_leader: false dest_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34"
I20260204 08:53:03.262308 2370 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 5 FOLLOWER]: Advancing to term 6
W20260204 08:53:03.265054 2199 leader_election.cc:336] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:03.267133 2370 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d083ecc9895345e5b06cd00ef0246c71 in term 6.
I20260204 08:53:03.268086 2200 leader_election.cc:304] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d083ecc9895345e5b06cd00ef0246c71, ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69
I20260204 08:53:03.269135 2419 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 6 FOLLOWER]: Leader election won for term 6
I20260204 08:53:03.269631 2419 raft_consensus.cc:697] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 6 LEADER]: Becoming Leader. State: Replica: d083ecc9895345e5b06cd00ef0246c71, State: Running, Role: LEADER
I20260204 08:53:03.270099 2419 consensus_queue.cc:237] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 100, Committed index: 100, Last appended: 5.100, Last appended by leader: 100, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:03.271808 2423 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d083ecc9895345e5b06cd00ef0246c71. Latest consensus state: current_term: 6 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:53:03.272147 2423 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: LEADER
I20260204 08:53:03.272894 2425 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260204 08:53:03.275700 2425 catalog_manager.cc:679] Loaded metadata for table default.table_f388b768c4b14290a90dd29962cc3fc4 [id=105a9508f80c4190a40ab4ab66af2872]
I20260204 08:53:03.276168 2425 catalog_manager.cc:679] Loaded metadata for table default.table_a638f2d4f2654854afdb0b065ee83f48 [id=12e17f7b128b4db99ae9041dba0602b5]
I20260204 08:53:03.276566 2425 catalog_manager.cc:679] Loaded metadata for table default.table_7bcbc3cfbc41406cba3ed79125a46e82 [id=180667e663144b7ea79638946d6c76cc]
I20260204 08:53:03.276909 2425 catalog_manager.cc:679] Loaded metadata for table default.table_e7fe420574dc40cdb70c4602f5a6e0db [id=19930831c99540688fb9a1e6ebfd8f82]
I20260204 08:53:03.277220 2425 catalog_manager.cc:679] Loaded metadata for table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]
I20260204 08:53:03.277632 2425 catalog_manager.cc:679] Loaded metadata for table default.table_97ec63740f2b4928af12e35b5eaa29cd [id=2a58ed39876b46aba637d263234e55d9]
I20260204 08:53:03.277982 2425 catalog_manager.cc:679] Loaded metadata for table default.table_f8273ef1a407421085918a8e3827f5f6 [id=2df47e721cd642e4b6fd5e61e172749c]
I20260204 08:53:03.278560 2425 catalog_manager.cc:679] Loaded metadata for table default.table_73e876d60c444a5daa316e4bf7158614 [id=42a606fb50bf4c58be67f67afab90d53]
I20260204 08:53:03.278959 2425 catalog_manager.cc:679] Loaded metadata for table default.table_1389d07a897b48439a706dce8abe591b [id=5a415846f1274434994325bb65b31911]
I20260204 08:53:03.279376 2425 catalog_manager.cc:679] Loaded metadata for table default.table_c97013dd16f84d8ba7cd528295f5ad49 [id=5f9f5b0850774ff9b8ecb93e141ff768]
I20260204 08:53:03.279686 2425 catalog_manager.cc:679] Loaded metadata for table default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807]
I20260204 08:53:03.280012 2425 catalog_manager.cc:679] Loaded metadata for table default.table_b4244312363c4346b110598eb01428a3 [id=7d4edb505f7940668b492a9d592a3e61]
I20260204 08:53:03.280382 2425 catalog_manager.cc:679] Loaded metadata for table default.table_9f93a995778f4fe6a4667cff0ef494b9 [id=893f125488f745a5b0bc9ec3d07da00b]
I20260204 08:53:03.280700 2425 catalog_manager.cc:679] Loaded metadata for table default.table_4f900fd574c849aba171e6a64bc43dbd [id=9427e77d6a3842a7a2665d99c8ebb99e]
I20260204 08:53:03.281021 2425 catalog_manager.cc:679] Loaded metadata for table default.table_477defa30c2c40e99360b505ef988a80 [id=b1e79546138b4dcb9c5a68977af78bf0]
I20260204 08:53:03.281349 2425 catalog_manager.cc:679] Loaded metadata for table default.table_f087faf331654fae99b16adcfb5f705f [id=b3b4145c845b417aa7582e41fb48b564]
I20260204 08:53:03.281649 2425 catalog_manager.cc:679] Loaded metadata for table default.table_9e950ede78084ef18f4e1051d9a42a90 [id=c626dc1c029e4b59902b4493f4e16a78]
I20260204 08:53:03.281973 2425 catalog_manager.cc:679] Loaded metadata for table default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294]
I20260204 08:53:03.282263 2425 catalog_manager.cc:679] Loaded metadata for table default.table_780ed695e5324a469614b35852a643cf [id=f29a6ab746244116a4815a632194b3ec]
I20260204 08:53:03.284354 2425 tablet_loader.cc:96] loaded metadata for tablet 043cdafd4b11463bae674695e7a74854 (table default.table_9f93a995778f4fe6a4667cff0ef494b9 [id=893f125488f745a5b0bc9ec3d07da00b])
I20260204 08:53:03.285073 2425 tablet_loader.cc:96] loaded metadata for tablet 2de648ce42af4a2fbd494c8cc3e38bb4 (table default.table_f388b768c4b14290a90dd29962cc3fc4 [id=105a9508f80c4190a40ab4ab66af2872])
I20260204 08:53:03.285379 2425 tablet_loader.cc:96] loaded metadata for tablet 350b048c73bd4208a7aa70b3521869e1 (table default.table_f8273ef1a407421085918a8e3827f5f6 [id=2df47e721cd642e4b6fd5e61e172749c])
I20260204 08:53:03.285655 2425 tablet_loader.cc:96] loaded metadata for tablet 35b597aac0264d36a44ae37c6ffb945e (table default.table_7bcbc3cfbc41406cba3ed79125a46e82 [id=180667e663144b7ea79638946d6c76cc])
I20260204 08:53:03.286089 2425 tablet_loader.cc:96] loaded metadata for tablet 4c26152dcc2147d6b0cb8d1242a7a73b (table default.table_477defa30c2c40e99360b505ef988a80 [id=b1e79546138b4dcb9c5a68977af78bf0])
I20260204 08:53:03.286360 2425 tablet_loader.cc:96] loaded metadata for tablet 5135d9b498314d9098339db68f64207d (table default.table_780ed695e5324a469614b35852a643cf [id=f29a6ab746244116a4815a632194b3ec])
I20260204 08:53:03.286652 2425 tablet_loader.cc:96] loaded metadata for tablet 52abcf65cd4840fdb7719ada0979c7a1 (table default.table_a638f2d4f2654854afdb0b065ee83f48 [id=12e17f7b128b4db99ae9041dba0602b5])
I20260204 08:53:03.287546 2425 tablet_loader.cc:96] loaded metadata for tablet 742c8a8c99464661873f292406ac5027 (table default.table_f087faf331654fae99b16adcfb5f705f [id=b3b4145c845b417aa7582e41fb48b564])
I20260204 08:53:03.287995 2425 tablet_loader.cc:96] loaded metadata for tablet 80f72b647ee643afabcc5cafa9c43b13 (table default.table_9e950ede78084ef18f4e1051d9a42a90 [id=c626dc1c029e4b59902b4493f4e16a78])
I20260204 08:53:03.288264 2425 tablet_loader.cc:96] loaded metadata for tablet 81f37ac4f50a440d8e7292bfa05a0a7d (table default.table_73e876d60c444a5daa316e4bf7158614 [id=42a606fb50bf4c58be67f67afab90d53])
I20260204 08:53:03.289031 2425 tablet_loader.cc:96] loaded metadata for tablet abef7b1f7bb6415186e9d5dc9b438604 (table default.table_b4244312363c4346b110598eb01428a3 [id=7d4edb505f7940668b492a9d592a3e61])
I20260204 08:53:03.289749 2425 tablet_loader.cc:96] loaded metadata for tablet c9ce4ab8ce1f48868b890df81e551eed (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421])
I20260204 08:53:03.290030 2425 tablet_loader.cc:96] loaded metadata for tablet cb72d1554d9b4c33b6cd7c01fad765db (table default.table_97ec63740f2b4928af12e35b5eaa29cd [id=2a58ed39876b46aba637d263234e55d9])
I20260204 08:53:03.290464 2425 tablet_loader.cc:96] loaded metadata for tablet d269b616e72f442e8a47f0c9192db8f8 (table default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807])
I20260204 08:53:03.290910 2425 tablet_loader.cc:96] loaded metadata for tablet deef771e18ac44a5b81f258fe8e6f3c4 (table default.table_4f900fd574c849aba171e6a64bc43dbd [id=9427e77d6a3842a7a2665d99c8ebb99e])
I20260204 08:53:03.291205 2425 tablet_loader.cc:96] loaded metadata for tablet e31d044c9bb14ceb8d637977966630b9 (table default.table_c97013dd16f84d8ba7cd528295f5ad49 [id=5f9f5b0850774ff9b8ecb93e141ff768])
I20260204 08:53:03.291504 2425 tablet_loader.cc:96] loaded metadata for tablet ebed8bb53e7947e08d0c960b13bacd8d (table default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294])
I20260204 08:53:03.291761 2425 tablet_loader.cc:96] loaded metadata for tablet eda7388ec12f4a7386823dadedaf2e2f (table default.table_1389d07a897b48439a706dce8abe591b [id=5a415846f1274434994325bb65b31911])
I20260204 08:53:03.292394 2425 tablet_loader.cc:96] loaded metadata for tablet fe0b2df895c643e7b69fb30a2b0b8e5c (table default.table_e7fe420574dc40cdb70c4602f5a6e0db [id=19930831c99540688fb9a1e6ebfd8f82])
I20260204 08:53:03.292671 2425 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260204 08:53:03.294003 2425 catalog_manager.cc:1269] Loaded cluster ID: 41c7773c62174f70835c9cfa744e1148
I20260204 08:53:03.294080 2425 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260204 08:53:03.295717 2425 catalog_manager.cc:1514] Loading token signing keys...
I20260204 08:53:03.296988 2425 catalog_manager.cc:6038] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Loaded TSK: 0
I20260204 08:53:03.297844 2425 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260204 08:53:03.299077 2425 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260204 08:53:03.300357 2425 catalog_manager.cc:5876] Last processed Hive Metastore notification event ID: 20
I20260204 08:53:03.357719 2214 master_service.cc:954] ReplaceTablet: received request to replace tablet fe0b2df895c643e7b69fb30a2b0b8e5c from {username='slave'} at 127.0.0.1:41510
I20260204 08:53:03.358810 2212 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41510:
table { table_name: "default.Table_477defa30c2c40e99360b505ef988a80" } modify_external_catalogs: true
I20260204 08:53:03.359105 2212 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41510:
table { table_name: "default.Table_477defa30c2c40e99360b505ef988a80" } modify_external_catalogs: true
I20260204 08:53:03.359448 2215 master_service.cc:954] ReplaceTablet: received request to replace tablet 4c26152dcc2147d6b0cb8d1242a7a73b from {username='slave'} at 127.0.0.1:41510
I20260204 08:53:03.359551 2210 master_service.cc:954] ReplaceTablet: received request to replace tablet d3c5509788ad4f8c858db788681b0920 from {username='slave'} at 127.0.0.1:41510
I20260204 08:53:03.360380 2210 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41510:
table { table_name: "default.Table_97ec63740f2b4928af12e35b5eaa29cd" } modify_external_catalogs: true
I20260204 08:53:03.360644 2210 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41510:
table { table_name: "default.Table_97ec63740f2b4928af12e35b5eaa29cd" } modify_external_catalogs: true
I20260204 08:53:03.360970 2209 master_service.cc:954] ReplaceTablet: received request to replace tablet c9ce4ab8ce1f48868b890df81e551eed from {username='slave'} at 127.0.0.1:41510
I20260204 08:53:03.361896 2211 master_service.cc:954] ReplaceTablet: received request to replace tablet fe0b2df895c643e7b69fb30a2b0b8e5c from {username='slave'} at 127.0.0.1:41510
I20260204 08:53:03.362618 2208 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41510:
name: "default.Table_76924b1fa5164c9f8261d67487aadc5f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:53:03.363576 2207 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41510:
name: "default.Table_270e36f84551482ead15da9f2b9be783"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:53:03.362682 2206 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41510:
table { table_name: "default.Table_7bcbc3cfbc41406cba3ed79125a46e82" } modify_external_catalogs: true
I20260204 08:53:03.364912 2206 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41510:
table { table_name: "default.Table_7bcbc3cfbc41406cba3ed79125a46e82" } modify_external_catalogs: true
I20260204 08:53:03.373857 2213 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41510:
name: "default.Table_81228fe5292c485e8e2260554ca79fd7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:53:03.375514 2370 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 6 FOLLOWER]: Refusing update from remote peer d083ecc9895345e5b06cd00ef0246c71: Log matching property violated. Preceding OpId in replica: term: 5 index: 100. Preceding OpId from leader: term: 6 index: 102. (index mismatch)
I20260204 08:53:03.376586 2419 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 101, Last known committed idx: 100, Time since last communication: 0.000s
W20260204 08:53:03.387804 2199 consensus_peers.cc:597] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 -> Peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Couldn't send request to peer c1e719e457e94a5da71929720411ae69. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
F20260204 08:53:03.393230 2271 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 23, previous 23; dumping first 4 out of 15 received events: NotificationEvent(eventId=21, eventTime=1770195168, eventType=DROP_TABLE, dbName=default, tableName=table_85f07b4907b24f1ca63791592455b59c, message={"server":"","servicePrincipal":"","db":"default","table":"table_85f07b4907b24f1ca63791592455b59c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_85f07b4907b24f1ca63791592455b59c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195167},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_85f07b4907b24f1ca63791592455b59c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_85f07b4907b24f1ca63791592455b59c\",\"kudu.table_id\":\"158d366d5ccf4b78b73903776a30cb35\",\"transient_lastDdlTime\":\"1770195167\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=22, eventTime=1770195168, eventType=CREATE_TABLE, dbName=default, tableName=table_97ec63740f2b4928af12e35b5eaa29cd, message={"server":"","servicePrincipal":"","db":"default","table":"table_97ec63740f2b4928af12e35b5eaa29cd","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_97ec63740f2b4928af12e35b5eaa29cd\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195168},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_97ec63740f2b4928af12e35b5eaa29cd\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_97ec63740f2b4928af12e35b5eaa29cd\",\"kudu.table_id\":\"2a58ed39876b46aba637d263234e55d9\",\"transient_lastDdlTime\":\"1770195168\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=23, eventTime=1770195168, eventType=DROP_TABLE, dbName=default, tableName=table_279632cf491c4198ad9b91a114c3fc33, message={"server":"","servicePrincipal":"","db":"default","table":"table_279632cf491c4198ad9b91a114c3fc33","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_279632cf491c4198ad9b91a114c3fc33\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195165},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_279632cf491c4198ad9b91a114c3fc33\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_279632cf491c4198ad9b91a114c3fc33\",\"kudu.table_id\":\"da8943eba12e426ba3bac81a64141294\",\"transient_lastDdlTime\":\"1770195165\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=23, eventTime=1770195168, eventType=CREATE_TABLE, dbName=default, tableName=table_477defa30c2c40e99360b505ef988a80, message={"server":"","servicePrincipal":"","db":"default","table":"table_477defa30c2c40e99360b505ef988a80","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_477defa30c2c40e99360b505ef988a80\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195168},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_477defa30c2c40e99360b505ef988a80\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_477defa30c2c40e99360b505ef988a80\",\"kudu.table_id\":\"b1e79546138b4dcb9c5a68977af78bf0\",\"transient_lastDdlTime\":\"1770195168\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f9f2871fdcd google::LogMessage::Fail() at ??:0
@ 0x7f9f28723b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f9f2871f7cc google::LogMessage::Flush() at ??:0
@ 0x7f9f2871fa11 google::LogMessage::~LogMessage() at ??:0
@ 0x7f9f3f1537f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260204 08:53:03.410181 2400 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d083ecc9895345e5b06cd00ef0246c71. Latest consensus state: current_term: 6 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:53:03.410480 2400 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f9f3f1520a1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f9f3f159d1c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260204 08:53:03.422451 2215 catalog_manager.cc:5941] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Sending DeleteTablet for 3 replicas of tablet 4c26152dcc2147d6b0cb8d1242a7a73b
I20260204 08:53:03.424539 2215 catalog_manager.cc:6724] ReplaceTablet: tablet 4c26152dcc2147d6b0cb8d1242a7a73b deleted and replaced by tablet 11d521bfd0db41f7b7d29f09709f5827
@ 0x7f9f3f159b99 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7f9f2b494f99 kudu::Thread::SuperviseThread() at ??:0
I20260204 08:53:03.444478 2209 catalog_manager.cc:5941] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Sending DeleteTablet for 3 replicas of tablet c9ce4ab8ce1f48868b890df81e551eed
I20260204 08:53:03.445075 2209 catalog_manager.cc:6724] ReplaceTablet: tablet c9ce4ab8ce1f48868b890df81e551eed deleted and replaced by tablet d5609ab66c4a4ba3afe829ac7c388531
I20260204 08:53:03.451754 2400 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:53:03.452009 2400 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:53:03.457360 1000 tablet_service.cc:1552] Processing DeleteTablet for tablet 4c26152dcc2147d6b0cb8d1242a7a73b with delete_type TABLET_DATA_DELETED (replaced by tablet 11d521bfd0db41f7b7d29f09709f5827) from {username='slave'} at 127.0.0.1:38454
I20260204 08:53:03.457396 1002 tablet_service.cc:1552] Processing DeleteTablet for tablet c9ce4ab8ce1f48868b890df81e551eed with delete_type TABLET_DATA_DELETED (replaced by tablet d5609ab66c4a4ba3afe829ac7c388531) from {username='slave'} at 127.0.0.1:38454
I20260204 08:53:03.460024 2214 catalog_manager.cc:5941] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Sending DeleteTablet for 3 replicas of tablet fe0b2df895c643e7b69fb30a2b0b8e5c
I20260204 08:53:03.461640 2214 catalog_manager.cc:6724] ReplaceTablet: tablet fe0b2df895c643e7b69fb30a2b0b8e5c deleted and replaced by tablet 3c6ed7e7133f45529ad48daa0bbcd70e
I20260204 08:53:03.462215 1138 tablet_service.cc:1552] Processing DeleteTablet for tablet 4c26152dcc2147d6b0cb8d1242a7a73b with delete_type TABLET_DATA_DELETED (replaced by tablet 11d521bfd0db41f7b7d29f09709f5827) from {username='slave'} at 127.0.0.1:56398
I20260204 08:53:03.462386 1140 tablet_service.cc:1552] Processing DeleteTablet for tablet c9ce4ab8ce1f48868b890df81e551eed with delete_type TABLET_DATA_DELETED (replaced by tablet d5609ab66c4a4ba3afe829ac7c388531) from {username='slave'} at 127.0.0.1:56398
I20260204 08:53:03.463129 1002 tablet_service.cc:1552] Processing DeleteTablet for tablet fe0b2df895c643e7b69fb30a2b0b8e5c with delete_type TABLET_DATA_DELETED (replaced by tablet 3c6ed7e7133f45529ad48daa0bbcd70e) from {username='slave'} at 127.0.0.1:38454
I20260204 08:53:03.463861 1137 tablet_service.cc:1552] Processing DeleteTablet for tablet fe0b2df895c643e7b69fb30a2b0b8e5c with delete_type TABLET_DATA_DELETED (replaced by tablet 3c6ed7e7133f45529ad48daa0bbcd70e) from {username='slave'} at 127.0.0.1:56398
I20260204 08:53:03.464386 2215 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41510:
name: "default.Table_8c3f05619b754ead910a58656d67b3b7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:53:03.464977 2419 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:53:03.465241 2419 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: LEADER
I20260204 08:53:03.465754 2442 tablet_replica.cc:333] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:53:03.465950 2442 raft_consensus.cc:2243] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:53:03.466318 2442 raft_consensus.cc:2272] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:53:03.467015 2442 ts_tablet_manager.cc:1916] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:53:03.474792 2440 tablet_replica.cc:333] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:53:03.475059 2440 raft_consensus.cc:2243] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:53:03.475057 2442 ts_tablet_manager.cc:1929] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:53:03.475196 2442 log.cc:1199] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/4c26152dcc2147d6b0cb8d1242a7a73b
I20260204 08:53:03.475298 2440 raft_consensus.cc:2272] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:53:03.475648 2442 ts_tablet_manager.cc:1950] T 4c26152dcc2147d6b0cb8d1242a7a73b P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:53:03.475894 2440 ts_tablet_manager.cc:1916] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:53:03.476738 2442 tablet_replica.cc:333] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:53:03.477324 2442 raft_consensus.cc:2243] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:53:03.477670 2442 raft_consensus.cc:2272] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:53:03.478278 2442 ts_tablet_manager.cc:1916] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:53:03.480762 2209 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41510:
table { table_name: "default.Table_9f93a995778f4fe6a4667cff0ef494b9" } new_table_name: "default.Table_b4e59736cf4b48b4b920d4ad64d8c522" modify_external_catalogs: true
I20260204 08:53:03.481216 2202 catalog_manager.cc:4985] TS fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737): tablet 4c26152dcc2147d6b0cb8d1242a7a73b (table default.table_477defa30c2c40e99360b505ef988a80 [id=b1e79546138b4dcb9c5a68977af78bf0]) successfully deleted
I20260204 08:53:03.481945 2214 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41510:
table { table_name: "default.Table_c97013dd16f84d8ba7cd528295f5ad49" } new_table_name: "default.Table_b3a5ec0b6897403bb998ef249c195d5f" modify_external_catalogs: true
I20260204 08:53:03.482347 2211 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41510:
table { table_name: "default.Table_67b2871860f446a1a5551b16cbbcec4d" } new_table_name: "default.Table_52f9941ea7774cceafc55a0edbdb526c" modify_external_catalogs: true
@ 0x7f9f2eaa16db start_thread at ??:0
I20260204 08:53:03.483323 1275 tablet_service.cc:1552] Processing DeleteTablet for tablet 4c26152dcc2147d6b0cb8d1242a7a73b with delete_type TABLET_DATA_DELETED (replaced by tablet 11d521bfd0db41f7b7d29f09709f5827) from {username='slave'} at 127.0.0.1:45794
I20260204 08:53:03.483599 1276 tablet_service.cc:1552] Processing DeleteTablet for tablet fe0b2df895c643e7b69fb30a2b0b8e5c with delete_type TABLET_DATA_DELETED (replaced by tablet 3c6ed7e7133f45529ad48daa0bbcd70e) from {username='slave'} at 127.0.0.1:45794
I20260204 08:53:03.483657 1273 tablet_service.cc:1552] Processing DeleteTablet for tablet c9ce4ab8ce1f48868b890df81e551eed with delete_type TABLET_DATA_DELETED (replaced by tablet d5609ab66c4a4ba3afe829ac7c388531) from {username='slave'} at 127.0.0.1:45794
@ 0x7f9f275d071f clone at ??:0
I20260204 08:53:03.488677 2440 ts_tablet_manager.cc:1929] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:53:03.488828 2440 log.cc:1199] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/4c26152dcc2147d6b0cb8d1242a7a73b
I20260204 08:53:03.489260 2440 ts_tablet_manager.cc:1950] T 4c26152dcc2147d6b0cb8d1242a7a73b P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:53:03.489440 2442 ts_tablet_manager.cc:1929] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260204 08:53:03.489583 2442 log.cc:1199] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/c9ce4ab8ce1f48868b890df81e551eed
I20260204 08:53:03.489960 2442 ts_tablet_manager.cc:1950] T c9ce4ab8ce1f48868b890df81e551eed P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:53:03.490490 2440 tablet_replica.cc:333] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:53:03.490708 2440 raft_consensus.cc:2243] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:53:03.491086 2440 raft_consensus.cc:2272] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:53:03.491670 2440 ts_tablet_manager.cc:1916] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:53:03.492095 2442 tablet_replica.cc:333] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070: stopping tablet replica
I20260204 08:53:03.492339 2442 raft_consensus.cc:2243] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:53:03.492558 2442 raft_consensus.cc:2272] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:53:03.493134 2442 ts_tablet_manager.cc:1916] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:53:03.496775 2444 tablet_replica.cc:333] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:53:03.497026 2444 raft_consensus.cc:2243] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:53:03.497279 2444 raft_consensus.cc:2272] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:53:03.497869 2444 ts_tablet_manager.cc:1916] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:53:03.499343 2442 ts_tablet_manager.cc:1929] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260204 08:53:03.499467 2442 log.cc:1199] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-1/wal/wals/fe0b2df895c643e7b69fb30a2b0b8e5c
I20260204 08:53:03.499590 2440 ts_tablet_manager.cc:1929] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260204 08:53:03.499756 2440 log.cc:1199] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/c9ce4ab8ce1f48868b890df81e551eed
I20260204 08:53:03.500196 2440 ts_tablet_manager.cc:1950] T c9ce4ab8ce1f48868b890df81e551eed P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:53:03.500757 2442 ts_tablet_manager.cc:1950] T fe0b2df895c643e7b69fb30a2b0b8e5c P fdea56c2d1f5499087b939ec9f9dc070: Deleting consensus metadata
I20260204 08:53:03.501900 2440 tablet_replica.cc:333] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae: stopping tablet replica
I20260204 08:53:03.502121 2440 raft_consensus.cc:2243] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:53:03.502337 2440 raft_consensus.cc:2272] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:53:03.502895 2440 ts_tablet_manager.cc:1916] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:53:03.504222 2444 ts_tablet_manager.cc:1929] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260204 08:53:03.504382 2444 log.cc:1199] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/4c26152dcc2147d6b0cb8d1242a7a73b
I20260204 08:53:03.504791 2444 ts_tablet_manager.cc:1950] T 4c26152dcc2147d6b0cb8d1242a7a73b P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:53:03.505832 2444 tablet_replica.cc:333] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:53:03.506067 2444 raft_consensus.cc:2243] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:53:03.506393 2444 raft_consensus.cc:2272] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:53:03.506976 2444 ts_tablet_manager.cc:1916] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:53:03.510044 2440 ts_tablet_manager.cc:1929] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260204 08:53:03.510192 2440 log.cc:1199] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-0/wal/wals/fe0b2df895c643e7b69fb30a2b0b8e5c
I20260204 08:53:03.510686 2440 ts_tablet_manager.cc:1950] T fe0b2df895c643e7b69fb30a2b0b8e5c P 6d471268b9e643e5bbbf8bde1644a5ae: Deleting consensus metadata
I20260204 08:53:03.513533 2444 ts_tablet_manager.cc:1929] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260204 08:53:03.513679 2444 log.cc:1199] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/fe0b2df895c643e7b69fb30a2b0b8e5c
I20260204 08:53:03.514122 2444 ts_tablet_manager.cc:1950] T fe0b2df895c643e7b69fb30a2b0b8e5c P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:53:03.515136 2444 tablet_replica.cc:333] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:53:03.515324 2444 raft_consensus.cc:2243] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260204 08:53:03.515528 2444 raft_consensus.cc:2272] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:53:03.516074 2444 ts_tablet_manager.cc:1916] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:53:03.534897 1067 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:53:03.538276 2360 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" instance_seqno: 1770195163347042) as {username='slave'} at 127.0.104.129:54325; Asking this server to re-register.
I20260204 08:53:03.539206 1067 heartbeater.cc:461] Registering TS with master...
I20260204 08:53:03.541658 2360 ts_manager.cc:194] Registered new tserver with Master: 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:53:03.542070 1205 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:53:03.546278 2360 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" instance_seqno: 1770195163874581) as {username='slave'} at 127.0.104.130:36155; Asking this server to re-register.
I20260204 08:53:03.549000 1205 heartbeater.cc:461] Registering TS with master...
I20260204 08:53:03.550779 1341 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:53:03.550810 2360 ts_manager.cc:194] Registered new tserver with Master: fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:53:03.552745 2360 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1234acd7d0db405085a5d95795007134" instance_seqno: 1770195164422899) as {username='slave'} at 127.0.104.131:56747; Asking this server to re-register.
I20260204 08:53:03.553097 2444 ts_tablet_manager.cc:1929] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260204 08:53:03.553232 2444 log.cc:1199] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/c9ce4ab8ce1f48868b890df81e551eed
I20260204 08:53:03.553611 1341 heartbeater.cc:461] Registering TS with master...
I20260204 08:53:03.553656 2444 ts_tablet_manager.cc:1950] T c9ce4ab8ce1f48868b890df81e551eed P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
I20260204 08:53:03.555670 2360 ts_manager.cc:194] Registered new tserver with Master: 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:53:03.703797 1230 connection.cc:537] server connection from 127.0.0.1:45794 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:53:03.703797 1096 connection.cc:537] server connection from 127.0.0.1:56398 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:53:03.704182 956 connection.cc:537] server connection from 127.0.0.1:38454 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:53:03.704631 1231 connection.cc:537] client connection to 127.0.104.189:37005 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:53:03.705230 1353 connection.cc:537] client connection to 127.0.104.189:37005 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:53:03.705946 1353 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.104.189:37005)
W20260204 08:53:03.706565 1353 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.104.189:37005)
W20260204 08:53:03.707134 1353 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.104.189:37005)
W20260204 08:53:03.707357 1353 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.104.189:37005)
I20260204 08:53:03.872822 1069 heartbeater.cc:499] Master 127.0.104.189:37005 was elected leader, sending a full tablet report...
I20260204 08:53:03.874336 1207 heartbeater.cc:499] Master 127.0.104.189:37005 was elected leader, sending a full tablet report...
I20260204 08:53:03.918413 2400 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:53:03.918632 2400 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:03.920367 2400 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:03.921346 2346 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:53:03.923133 2346 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:53:03.923664 2344 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:03.923808 2344 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:03.924594 2400 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260204 08:53:04.057243 2329 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:04.057432 2329 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:53:04.058144 2329 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:53:04.059065 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:53:04.059274 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:53:04.059383 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:53:04.059643 2329 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:53:04.603168 2456 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:53:04.603371 2456 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:04.604028 2456 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:04.606154 2346 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:53:04.606376 2344 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:04.606493 2344 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:04.606791 2456 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260204 08:53:05.225805 2462 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:53:05.226019 2462 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:05.226694 2462 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:05.228711 2344 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:53:05.228876 2346 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:53:05.229050 2346 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:05.229311 2462 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260204 08:53:06.340210 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
/tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:37005
--webserver_interface=127.0.104.189
--webserver_port=43227
--builtin_ntp_servers=127.0.104.148:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260204 08:53:06.695879 2468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260204 08:53:06.696221 2468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260204 08:53:06.696346 2468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260204 08:53:06.705379 2468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260204 08:53:06.705492 2468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260204 08:53:06.705547 2468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260204 08:53:06.705597 2468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260204 08:53:06.716251 2468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.189:37005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.104.189
--webserver_port=43227
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0efbebd487e7d0c7f21ab0d5a91e0fa70046af83
build type FASTDEBUG
built by None at 04 Feb 2026 08:43:20 UTC on 24a791456cd2
build id 10340
ASAN enabled
I20260204 08:53:06.717921 2468 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260204 08:53:06.719831 2468 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260204 08:53:06.728840 2473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:53:06.729214 2474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:53:06.728761 2476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260204 08:53:06.728912 2478 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:53:06.729622 2478 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:06.729990 2468 server_base.cc:1060] running on GCE node
I20260204 08:53:06.730460 2478 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
I20260204 08:53:06.731058 2468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260204 08:53:06.733189 2468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
W20260204 08:53:06.733201 2346 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:53:06.733525 2344 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:06.733660 2344 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:06.733948 2478 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260204 08:53:06.734514 2468 hybrid_clock.cc:648] HybridClock initialized: now 1770195186734435 us; error 61 us; skew 500 ppm
I20260204 08:53:06.734951 2468 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260204 08:53:06.738245 2468 webserver.cc:492] Webserver started at http://127.0.104.189:43227/ using document root <none> and password file <none>
I20260204 08:53:06.739108 2468 fs_manager.cc:362] Metadata directory not provided
I20260204 08:53:06.739213 2468 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260204 08:53:06.745561 2468 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260204 08:53:06.748919 2486 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260204 08:53:06.750507 2468 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260204 08:53:06.750682 2468 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data,/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
uuid: "d083ecc9895345e5b06cd00ef0246c71"
format_stamp: "Formatted at 2026-02-04 08:52:41 on dist-test-slave-vp4h"
I20260204 08:53:06.751485 2468 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260204 08:53:06.786427 2468 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260204 08:53:06.787334 2468 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260204 08:53:06.787628 2468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260204 08:53:06.808701 2468 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.189:37005
I20260204 08:53:06.808745 2537 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.189:37005 every 8 connection(s)
I20260204 08:53:06.810334 2468 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-1/data/info.pb
I20260204 08:53:06.818007 2538 sys_catalog.cc:263] Verifying existing consensus state
I20260204 08:53:06.820259 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu as pid 2468
I20260204 08:53:06.821906 2538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Bootstrap starting.
I20260204 08:53:06.841404 2538 log.cc:826] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Log is configured to *not* fsync() on all Append() calls
I20260204 08:53:06.890262 1069 heartbeater.cc:344] Connected to a master server at 127.0.104.189:37005
I20260204 08:53:06.890515 1069 heartbeater.cc:499] Master 127.0.104.189:37005 was elected leader, sending a full tablet report...
I20260204 08:53:06.891897 1207 heartbeater.cc:344] Connected to a master server at 127.0.104.189:37005
I20260204 08:53:06.892128 1207 heartbeater.cc:499] Master 127.0.104.189:37005 was elected leader, sending a full tablet report...
I20260204 08:53:06.927619 2538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Bootstrap replayed 1/1 log segments. Stats: ops{read=104 overwritten=0 applied=104 ignored=0} inserts{seen=70 ignored=0} mutations{seen=89 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260204 08:53:06.928143 2538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Bootstrap complete.
I20260204 08:53:06.933636 2538 raft_consensus.cc:359] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:06.934378 2538 raft_consensus.cc:740] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: d083ecc9895345e5b06cd00ef0246c71, State: Initialized, Role: FOLLOWER
I20260204 08:53:06.935077 2538 consensus_queue.cc:260] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 104, Last appended: 6.104, Last appended by leader: 104, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:06.936040 2543 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:53:06.937189 2543 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:53:06.937575 2538 sys_catalog.cc:565] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: configured and running, proceeding with master startup.
I20260204 08:53:06.955874 2557 catalog_manager.cc:1269] Loaded cluster ID: 41c7773c62174f70835c9cfa744e1148
I20260204 08:53:06.956048 2557 catalog_manager.cc:1562] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: loading cluster ID for follower catalog manager: success
I20260204 08:53:06.961781 2557 catalog_manager.cc:1584] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: acquiring CA information for follower catalog manager: success
I20260204 08:53:06.966327 2557 catalog_manager.cc:1612] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260204 08:53:06.982344 2468 master_runner.cc:186] Error getting master registration for 127.0.104.188:38981: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111),
I20260204 08:53:06.986140 2468 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:53:07.219345 1343 heartbeater.cc:344] Connected to a master server at 127.0.104.189:37005
I20260204 08:53:07.222048 2503 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1234acd7d0db405085a5d95795007134" instance_seqno: 1770195164422899) as {username='slave'} at 127.0.104.131:43979; Asking this server to re-register.
I20260204 08:53:07.222877 1343 heartbeater.cc:461] Registering TS with master...
I20260204 08:53:07.225138 2503 ts_manager.cc:194] Registered new tserver with Master: 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:53:07.395152 1069 heartbeater.cc:499] Master 127.0.104.189:37005 was elected leader, sending a full tablet report...
I20260204 08:53:07.395413 1207 heartbeater.cc:499] Master 127.0.104.189:37005 was elected leader, sending a full tablet report...
I20260204 08:53:07.398272 2503 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" instance_seqno: 1770195163347042) as {username='slave'} at 127.0.104.129:37447; Asking this server to re-register.
I20260204 08:53:07.398399 2502 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" instance_seqno: 1770195163874581) as {username='slave'} at 127.0.104.130:51301; Asking this server to re-register.
I20260204 08:53:07.399114 1069 heartbeater.cc:461] Registering TS with master...
I20260204 08:53:07.399281 1207 heartbeater.cc:461] Registering TS with master...
I20260204 08:53:07.401057 2502 ts_manager.cc:194] Registered new tserver with Master: 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:53:07.401130 2503 ts_manager.cc:194] Registered new tserver with Master: fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:53:07.455163 2562 raft_consensus.cc:493] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:07.455446 2562 raft_consensus.cc:515] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:07.457465 2562 leader_election.cc:290] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
I20260204 08:53:07.458002 2370 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d083ecc9895345e5b06cd00ef0246c71" candidate_term: 7 candidate_status { last_received { term: 6 index: 104 } } ignore_live_leader: false dest_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" is_pre_election: true
W20260204 08:53:07.458161 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:07.458320 2370 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d083ecc9895345e5b06cd00ef0246c71 in term 6.
I20260204 08:53:07.458942 2488 leader_election.cc:304] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d083ecc9895345e5b06cd00ef0246c71, ebeeb5d4823b41b3a8e37546ff2f6e34; no voters:
I20260204 08:53:07.459334 2562 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260204 08:53:07.459470 2562 raft_consensus.cc:493] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260204 08:53:07.459576 2562 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 6 FOLLOWER]: Advancing to term 7
W20260204 08:53:07.459744 2487 leader_election.cc:336] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:07.463222 2562 raft_consensus.cc:515] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:07.463840 2562 leader_election.cc:290] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 7 election: Requested vote from peers ebeeb5d4823b41b3a8e37546ff2f6e34 (127.0.104.190:42471), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
I20260204 08:53:07.464412 2370 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d083ecc9895345e5b06cd00ef0246c71" candidate_term: 7 candidate_status { last_received { term: 6 index: 104 } } ignore_live_leader: false dest_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34"
I20260204 08:53:07.464634 2370 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 6 FOLLOWER]: Advancing to term 7
W20260204 08:53:07.465260 2487 leader_election.cc:336] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:07.466625 2370 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d083ecc9895345e5b06cd00ef0246c71 in term 7.
I20260204 08:53:07.467090 2488 leader_election.cc:304] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d083ecc9895345e5b06cd00ef0246c71, ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69
I20260204 08:53:07.467463 2562 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 7 FOLLOWER]: Leader election won for term 7
I20260204 08:53:07.467854 2562 raft_consensus.cc:697] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [term 7 LEADER]: Becoming Leader. State: Replica: d083ecc9895345e5b06cd00ef0246c71, State: Running, Role: LEADER
I20260204 08:53:07.468338 2562 consensus_queue.cc:237] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 104, Committed index: 104, Last appended: 6.104, Last appended by leader: 104, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:07.471032 2564 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d083ecc9895345e5b06cd00ef0246c71. Latest consensus state: current_term: 7 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:53:07.471253 2564 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: LEADER
I20260204 08:53:07.471922 2566 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260204 08:53:07.474939 2566 catalog_manager.cc:679] Loaded metadata for table default.table_f388b768c4b14290a90dd29962cc3fc4 [id=105a9508f80c4190a40ab4ab66af2872]
I20260204 08:53:07.475404 2566 catalog_manager.cc:679] Loaded metadata for table default.table_a638f2d4f2654854afdb0b065ee83f48 [id=12e17f7b128b4db99ae9041dba0602b5]
I20260204 08:53:07.475729 2566 catalog_manager.cc:679] Loaded metadata for table default.table_7bcbc3cfbc41406cba3ed79125a46e82 [id=180667e663144b7ea79638946d6c76cc]
I20260204 08:53:07.476027 2566 catalog_manager.cc:679] Loaded metadata for table default.table_e7fe420574dc40cdb70c4602f5a6e0db [id=19930831c99540688fb9a1e6ebfd8f82]
I20260204 08:53:07.476328 2566 catalog_manager.cc:679] Loaded metadata for table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421]
I20260204 08:53:07.476656 2566 catalog_manager.cc:679] Loaded metadata for table default.table_97ec63740f2b4928af12e35b5eaa29cd [id=2a58ed39876b46aba637d263234e55d9]
I20260204 08:53:07.476933 2566 catalog_manager.cc:679] Loaded metadata for table default.table_f8273ef1a407421085918a8e3827f5f6 [id=2df47e721cd642e4b6fd5e61e172749c]
I20260204 08:53:07.477393 2566 catalog_manager.cc:679] Loaded metadata for table default.table_73e876d60c444a5daa316e4bf7158614 [id=42a606fb50bf4c58be67f67afab90d53]
I20260204 08:53:07.477698 2566 catalog_manager.cc:679] Loaded metadata for table default.table_1389d07a897b48439a706dce8abe591b [id=5a415846f1274434994325bb65b31911]
I20260204 08:53:07.478015 2566 catalog_manager.cc:679] Loaded metadata for table default.table_c97013dd16f84d8ba7cd528295f5ad49 [id=5f9f5b0850774ff9b8ecb93e141ff768]
I20260204 08:53:07.478286 2566 catalog_manager.cc:679] Loaded metadata for table default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807]
I20260204 08:53:07.478561 2566 catalog_manager.cc:679] Loaded metadata for table default.table_b4244312363c4346b110598eb01428a3 [id=7d4edb505f7940668b492a9d592a3e61]
I20260204 08:53:07.478835 2566 catalog_manager.cc:679] Loaded metadata for table default.table_9f93a995778f4fe6a4667cff0ef494b9 [id=893f125488f745a5b0bc9ec3d07da00b]
I20260204 08:53:07.479120 2566 catalog_manager.cc:679] Loaded metadata for table default.table_4f900fd574c849aba171e6a64bc43dbd [id=9427e77d6a3842a7a2665d99c8ebb99e]
I20260204 08:53:07.479394 2566 catalog_manager.cc:679] Loaded metadata for table default.table_477defa30c2c40e99360b505ef988a80 [id=b1e79546138b4dcb9c5a68977af78bf0]
I20260204 08:53:07.479668 2566 catalog_manager.cc:679] Loaded metadata for table default.table_f087faf331654fae99b16adcfb5f705f [id=b3b4145c845b417aa7582e41fb48b564]
I20260204 08:53:07.479933 2566 catalog_manager.cc:679] Loaded metadata for table default.table_9e950ede78084ef18f4e1051d9a42a90 [id=c626dc1c029e4b59902b4493f4e16a78]
I20260204 08:53:07.480392 2566 catalog_manager.cc:679] Loaded metadata for table default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294]
I20260204 08:53:07.480720 2566 catalog_manager.cc:679] Loaded metadata for table default.table_780ed695e5324a469614b35852a643cf [id=f29a6ab746244116a4815a632194b3ec]
I20260204 08:53:07.482947 2566 tablet_loader.cc:96] loaded metadata for tablet 043cdafd4b11463bae674695e7a74854 (table default.table_9f93a995778f4fe6a4667cff0ef494b9 [id=893f125488f745a5b0bc9ec3d07da00b])
I20260204 08:53:07.483495 2566 tablet_loader.cc:96] loaded metadata for tablet 11d521bfd0db41f7b7d29f09709f5827 (table default.table_477defa30c2c40e99360b505ef988a80 [id=b1e79546138b4dcb9c5a68977af78bf0])
I20260204 08:53:07.483927 2566 tablet_loader.cc:96] loaded metadata for tablet 2de648ce42af4a2fbd494c8cc3e38bb4 (table default.table_f388b768c4b14290a90dd29962cc3fc4 [id=105a9508f80c4190a40ab4ab66af2872])
I20260204 08:53:07.484236 2566 tablet_loader.cc:96] loaded metadata for tablet 350b048c73bd4208a7aa70b3521869e1 (table default.table_f8273ef1a407421085918a8e3827f5f6 [id=2df47e721cd642e4b6fd5e61e172749c])
I20260204 08:53:07.484555 2566 tablet_loader.cc:96] loaded metadata for tablet 35b597aac0264d36a44ae37c6ffb945e (table default.table_7bcbc3cfbc41406cba3ed79125a46e82 [id=180667e663144b7ea79638946d6c76cc])
I20260204 08:53:07.484779 2566 tablet_loader.cc:96] loaded metadata for tablet 3c6ed7e7133f45529ad48daa0bbcd70e (table default.table_e7fe420574dc40cdb70c4602f5a6e0db [id=19930831c99540688fb9a1e6ebfd8f82])
I20260204 08:53:07.485364 2566 tablet_loader.cc:96] loaded metadata for tablet 5135d9b498314d9098339db68f64207d (table default.table_780ed695e5324a469614b35852a643cf [id=f29a6ab746244116a4815a632194b3ec])
I20260204 08:53:07.485617 2566 tablet_loader.cc:96] loaded metadata for tablet 52abcf65cd4840fdb7719ada0979c7a1 (table default.table_a638f2d4f2654854afdb0b065ee83f48 [id=12e17f7b128b4db99ae9041dba0602b5])
I20260204 08:53:07.486528 2566 tablet_loader.cc:96] loaded metadata for tablet 742c8a8c99464661873f292406ac5027 (table default.table_f087faf331654fae99b16adcfb5f705f [id=b3b4145c845b417aa7582e41fb48b564])
I20260204 08:53:07.486932 2566 tablet_loader.cc:96] loaded metadata for tablet 80f72b647ee643afabcc5cafa9c43b13 (table default.table_9e950ede78084ef18f4e1051d9a42a90 [id=c626dc1c029e4b59902b4493f4e16a78])
I20260204 08:53:07.487428 2566 tablet_loader.cc:96] loaded metadata for tablet 81f37ac4f50a440d8e7292bfa05a0a7d (table default.table_73e876d60c444a5daa316e4bf7158614 [id=42a606fb50bf4c58be67f67afab90d53])
I20260204 08:53:07.488181 2566 tablet_loader.cc:96] loaded metadata for tablet abef7b1f7bb6415186e9d5dc9b438604 (table default.table_b4244312363c4346b110598eb01428a3 [id=7d4edb505f7940668b492a9d592a3e61])
I20260204 08:53:07.489060 2566 tablet_loader.cc:96] loaded metadata for tablet cb72d1554d9b4c33b6cd7c01fad765db (table default.table_97ec63740f2b4928af12e35b5eaa29cd [id=2a58ed39876b46aba637d263234e55d9])
I20260204 08:53:07.489495 2566 tablet_loader.cc:96] loaded metadata for tablet d269b616e72f442e8a47f0c9192db8f8 (table default.table_732f1b588b344e49b2f77e17e6551853 [id=76307bf8c30c4654abb6343e24eed807])
I20260204 08:53:07.489877 2566 tablet_loader.cc:96] loaded metadata for tablet d5609ab66c4a4ba3afe829ac7c388531 (table default.table_67b2871860f446a1a5551b16cbbcec4d [id=242a7b5626014adf9859874c46290421])
I20260204 08:53:07.490151 2566 tablet_loader.cc:96] loaded metadata for tablet deef771e18ac44a5b81f258fe8e6f3c4 (table default.table_4f900fd574c849aba171e6a64bc43dbd [id=9427e77d6a3842a7a2665d99c8ebb99e])
I20260204 08:53:07.490425 2566 tablet_loader.cc:96] loaded metadata for tablet e31d044c9bb14ceb8d637977966630b9 (table default.table_c97013dd16f84d8ba7cd528295f5ad49 [id=5f9f5b0850774ff9b8ecb93e141ff768])
I20260204 08:53:07.490688 2566 tablet_loader.cc:96] loaded metadata for tablet ebed8bb53e7947e08d0c960b13bacd8d (table default.table_279632cf491c4198ad9b91a114c3fc33 [id=da8943eba12e426ba3bac81a64141294])
I20260204 08:53:07.490970 2566 tablet_loader.cc:96] loaded metadata for tablet eda7388ec12f4a7386823dadedaf2e2f (table default.table_1389d07a897b48439a706dce8abe591b [id=5a415846f1274434994325bb65b31911])
I20260204 08:53:07.491542 2566 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260204 08:53:07.492988 2566 catalog_manager.cc:1269] Loaded cluster ID: 41c7773c62174f70835c9cfa744e1148
I20260204 08:53:07.493072 2566 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260204 08:53:07.494724 2566 catalog_manager.cc:1514] Loading token signing keys...
I20260204 08:53:07.496034 2566 catalog_manager.cc:6038] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Loaded TSK: 0
I20260204 08:53:07.496971 2566 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260204 08:53:07.498216 2566 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260204 08:53:07.499477 2566 catalog_manager.cc:5876] Last processed Hive Metastore notification event ID: 20
I20260204 08:53:07.506251 2503 master_service.cc:954] ReplaceTablet: received request to replace tablet 742c8a8c99464661873f292406ac5027 from {username='slave'} at 127.0.0.1:48872
I20260204 08:53:07.506742 2500 master_service.cc:954] ReplaceTablet: received request to replace tablet 2de648ce42af4a2fbd494c8cc3e38bb4 from {username='slave'} at 127.0.0.1:48872
I20260204 08:53:07.507462 2497 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48872:
name: "default.Table_14079082888e40fc818c5b295ee4da92"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:53:07.508630 2498 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48872:
name: "default.Table_8c0106d78fea4bd9846bde27656144cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:53:07.509238 2499 master_service.cc:954] ReplaceTablet: received request to replace tablet d269b616e72f442e8a47f0c9192db8f8 from {username='slave'} at 127.0.0.1:48872
I20260204 08:53:07.509119 2496 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48872:
name: "default.Table_2b92e77d55da4edb9ab442d799aa9e06"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:53:07.510207 2501 master_service.cc:954] ReplaceTablet: received request to replace tablet eda7388ec12f4a7386823dadedaf2e2f from {username='slave'} at 127.0.0.1:48872
I20260204 08:53:07.509929 2495 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48872:
name: "default.Table_27cce8dee4984465be213744abfadc9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:53:07.511075 2502 master_service.cc:954] ReplaceTablet: received request to replace tablet e31d044c9bb14ceb8d637977966630b9 from {username='slave'} at 127.0.0.1:48872
I20260204 08:53:07.510632 2494 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48872:
name: "default.Table_85f07b4907b24f1ca63791592455b59c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260204 08:53:07.518534 2370 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Refusing update from remote peer d083ecc9895345e5b06cd00ef0246c71: Log matching property violated. Preceding OpId in replica: term: 6 index: 104. Preceding OpId from leader: term: 7 index: 107. (index mismatch)
W20260204 08:53:07.519302 2487 consensus_peers.cc:597] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 -> Peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Couldn't send request to peer c1e719e457e94a5da71929720411ae69. Status: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260204 08:53:07.520364 2564 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 105, Last known committed idx: 104, Time since last communication: 0.000s
F20260204 08:53:07.520484 2556 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 23, previous 23; dumping first 4 out of 15 received events: NotificationEvent(eventId=21, eventTime=1770195168, eventType=DROP_TABLE, dbName=default, tableName=table_85f07b4907b24f1ca63791592455b59c, message={"server":"","servicePrincipal":"","db":"default","table":"table_85f07b4907b24f1ca63791592455b59c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_85f07b4907b24f1ca63791592455b59c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195167},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_85f07b4907b24f1ca63791592455b59c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_85f07b4907b24f1ca63791592455b59c\",\"kudu.table_id\":\"158d366d5ccf4b78b73903776a30cb35\",\"transient_lastDdlTime\":\"1770195167\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=22, eventTime=1770195168, eventType=CREATE_TABLE, dbName=default, tableName=table_97ec63740f2b4928af12e35b5eaa29cd, message={"server":"","servicePrincipal":"","db":"default","table":"table_97ec63740f2b4928af12e35b5eaa29cd","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_97ec63740f2b4928af12e35b5eaa29cd\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195168},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_97ec63740f2b4928af12e35b5eaa29cd\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_97ec63740f2b4928af12e35b5eaa29cd\",\"kudu.table_id\":\"2a58ed39876b46aba637d263234e55d9\",\"transient_lastDdlTime\":\"1770195168\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=23, eventTime=1770195168, eventType=DROP_TABLE, dbName=default, tableName=table_279632cf491c4198ad9b91a114c3fc33, message={"server":"","servicePrincipal":"","db":"default","table":"table_279632cf491c4198ad9b91a114c3fc33","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_279632cf491c4198ad9b91a114c3fc33\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195165},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_279632cf491c4198ad9b91a114c3fc33\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_279632cf491c4198ad9b91a114c3fc33\",\"kudu.table_id\":\"da8943eba12e426ba3bac81a64141294\",\"transient_lastDdlTime\":\"1770195165\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=23, eventTime=1770195168, eventType=CREATE_TABLE, dbName=default, tableName=table_477defa30c2c40e99360b505ef988a80, message={"server":"","servicePrincipal":"","db":"default","table":"table_477defa30c2c40e99360b505ef988a80","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_477defa30c2c40e99360b505ef988a80\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1770195168},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/warehouse/table_477defa30c2c40e99360b505ef988a80\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_477defa30c2c40e99360b505ef988a80\",\"kudu.table_id\":\"b1e79546138b4dcb9c5a68977af78bf0\",\"transient_lastDdlTime\":\"1770195168\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"41c7773c62174f70835c9cfa744e1148\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1770195168,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f28da0fddcd google::LogMessage::Fail() at ??:0
@ 0x7f28da101b93 google::LogMessage::SendToLog() at ??:0
I20260204 08:53:07.525363 2568 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d083ecc9895345e5b06cd00ef0246c71. Latest consensus state: current_term: 7 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:53:07.525642 2568 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f28da0fd7cc google::LogMessage::Flush() at ??:0
@ 0x7f28da0fda11 google::LogMessage::~LogMessage() at ??:0
I20260204 08:53:07.528484 2567 mvcc.cc:204] Tried to move back new op lower bound from 7250719488061014016 to 7250719487873912832. Current Snapshot: MvccSnapshot[applied={T|T < 7250719488061014016}]
@ 0x7f28f0b317f4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260204 08:53:07.556962 2568 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:53:07.557257 2568 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:53:07.560113 2500 catalog_manager.cc:5941] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Sending DeleteTablet for 3 replicas of tablet 2de648ce42af4a2fbd494c8cc3e38bb4
I20260204 08:53:07.561026 2503 catalog_manager.cc:5941] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Sending DeleteTablet for 3 replicas of tablet 742c8a8c99464661873f292406ac5027
I20260204 08:53:07.561877 2499 catalog_manager.cc:5941] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Sending DeleteTablet for 3 replicas of tablet d269b616e72f442e8a47f0c9192db8f8
@ 0x7f28f0b300a1 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260204 08:53:07.562125 2503 catalog_manager.cc:6724] ReplaceTablet: tablet 742c8a8c99464661873f292406ac5027 deleted and replaced by tablet 5fd8bffb1e074598a892680150a3a3fa
I20260204 08:53:07.562304 2499 catalog_manager.cc:6724] ReplaceTablet: tablet d269b616e72f442e8a47f0c9192db8f8 deleted and replaced by tablet 8ceef6dea44243c8b6da3d22d83af219
I20260204 08:53:07.563077 2500 catalog_manager.cc:6724] ReplaceTablet: tablet 2de648ce42af4a2fbd494c8cc3e38bb4 deleted and replaced by tablet 5b43a01114cf49a9846aacb60f894467
@ 0x7f28f0b37d1c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f28f0b37b99 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xa8970d std::function<>::operator()() at ??:?
@ 0x7f28dce72f99 kudu::Thread::SuperviseThread() at ??:0
I20260204 08:53:07.585402 2501 catalog_manager.cc:5941] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Sending DeleteTablet for 3 replicas of tablet eda7388ec12f4a7386823dadedaf2e2f
I20260204 08:53:07.585580 2564 sys_catalog.cc:455] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "d083ecc9895345e5b06cd00ef0246c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:53:07.585723 1275 tablet_service.cc:1552] Processing DeleteTablet for tablet 2de648ce42af4a2fbd494c8cc3e38bb4 with delete_type TABLET_DATA_DELETED (replaced by tablet 5b43a01114cf49a9846aacb60f894467) from {username='slave'} at 127.0.0.1:42858
I20260204 08:53:07.585839 2564 sys_catalog.cc:458] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71 [sys.catalog]: This master's current role is: LEADER
I20260204 08:53:07.585942 2501 catalog_manager.cc:6724] ReplaceTablet: tablet eda7388ec12f4a7386823dadedaf2e2f deleted and replaced by tablet d3aede385b314bfb9d4161b306ff6174
I20260204 08:53:07.586283 2502 catalog_manager.cc:5941] T 00000000000000000000000000000000 P d083ecc9895345e5b06cd00ef0246c71: Sending DeleteTablet for 3 replicas of tablet e31d044c9bb14ceb8d637977966630b9
I20260204 08:53:07.586374 2501 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48872:
table { table_name: "default.Table_732f1b588b344e49b2f77e17e6551853" } new_table_name: "default.Table_ae16851d63e6436692ececda3e21408e" modify_external_catalogs: true
I20260204 08:53:07.587847 2502 catalog_manager.cc:6724] ReplaceTablet: tablet e31d044c9bb14ceb8d637977966630b9 deleted and replaced by tablet bafa167f2ed741518e2f817d76229502
I20260204 08:53:07.588253 2502 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48872:
name: "default.Table_76924b1fa5164c9f8261d67487aadc5f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f28e047f6db start_thread at ??:0
I20260204 08:53:07.588630 2499 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48872:
table { table_name: "default.Table_e7fe420574dc40cdb70c4602f5a6e0db" } new_table_name: "default.Table_2cf92b501b134cc2a2c8be92b1530908" modify_external_catalogs: true
I20260204 08:53:07.588866 2500 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48872:
table { table_name: "default.Table_b4244312363c4346b110598eb01428a3" } modify_external_catalogs: true
I20260204 08:53:07.589105 2500 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48872:
table { table_name: "default.Table_b4244312363c4346b110598eb01428a3" } modify_external_catalogs: true
@ 0x7f28d8fae71f clone at ??:0
I20260204 08:53:07.591071 2587 tablet_replica.cc:333] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134: stopping tablet replica
I20260204 08:53:07.591291 2587 raft_consensus.cc:2243] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [term 1 LEADER]: Raft consensus shutting down.
I20260204 08:53:07.591717 2587 raft_consensus.cc:2272] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260204 08:53:07.592521 2587 ts_tablet_manager.cc:1916] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260204 08:53:07.599212 2587 ts_tablet_manager.cc:1929] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260204 08:53:07.599377 2587 log.cc:1199] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134: Deleting WAL directory at /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/ts-2/wal/wals/2de648ce42af4a2fbd494c8cc3e38bb4
I20260204 08:53:07.599799 2587 ts_tablet_manager.cc:1950] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 1234acd7d0db405085a5d95795007134: Deleting consensus metadata
W20260204 08:53:07.759186 1231 connection.cc:537] server connection from 127.0.0.1:42858 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:53:07.759479 1231 connection.cc:537] client connection to 127.0.104.189:37005 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260204 08:53:07.760102 1353 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.104.189:37005)
W20260204 08:53:07.760416 1353 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.104.189:37005)
W20260204 08:53:07.760776 1353 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.104.189:37005)
W20260204 08:53:07.761198 2586 negotiation.cc:337] Failed RPC negotiation. Trace:
0204 08:53:07.574212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42942 (local address 127.0.104.129:36675)
0204 08:53:07.591291 (+ 17079us) server_negotiation.cc:207] Beginning negotiation
0204 08:53:07.591298 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0204 08:53:07.760086 (+168788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42942: BlockingRecv error: recv got EOF from 127.0.0.1:42942 (error 108)
Metrics: {"server-negotiator.queue_time_us":16896,"thread_start_us":16744,"threads_started":1}
W20260204 08:53:07.770364 1353 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.104.189:37005)
I20260204 08:53:08.028193 2568 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:53:08.028440 2568 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:08.029096 2568 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:08.031229 2344 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:53:08.031473 2346 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:53:08.031591 2346 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:08.031872 2568 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:53:08.260699 2328 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:53:08.260959 2328 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:53:08.356212 2329 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:08.356482 2329 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:53:08.357096 2329 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:53:08.358186 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:53:08.358430 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:53:08.358522 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:53:08.358804 2329 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:53:08.588330 2601 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d083ecc9895345e5b06cd00ef0246c71)
I20260204 08:53:08.588519 2601 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:08.589238 2601 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:08.590822 2346 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:53:08.591029 2344 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:08.591174 2344 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:08.591464 2601 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:53:08.993513 418 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu pid 2328
I20260204 08:53:09.000826 2390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260204 08:53:09.115401 418 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu with pid 2328
I20260204 08:53:09.870605 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
/tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:42471
--webserver_interface=127.0.104.190
--webserver_port=40787
--builtin_ntp_servers=127.0.104.148:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260204 08:53:10.073122 2604 raft_consensus.cc:493] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1234acd7d0db405085a5d95795007134)
I20260204 08:53:10.073371 2604 raft_consensus.cc:515] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:53:10.074151 2604 leader_election.cc:290] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:53:10.074856 1158 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2de648ce42af4a2fbd494c8cc3e38bb4" candidate_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fdea56c2d1f5499087b939ec9f9dc070" is_pre_election: true
W20260204 08:53:10.075372 958 leader_election.cc:343] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4
I20260204 08:53:10.075654 958 leader_election.cc:304] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae; no voters: 1234acd7d0db405085a5d95795007134, fdea56c2d1f5499087b939ec9f9dc070
I20260204 08:53:10.075903 2604 raft_consensus.cc:2749] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260204 08:53:10.112126 2329 raft_consensus.cc:493] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1234acd7d0db405085a5d95795007134)
I20260204 08:53:10.112406 2329 raft_consensus.cc:515] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:53:10.113139 2329 leader_election.cc:290] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:53:10.113718 1019 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2de648ce42af4a2fbd494c8cc3e38bb4" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" is_pre_election: true
I20260204 08:53:10.113965 1019 raft_consensus.cc:2468] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 1.
I20260204 08:53:10.114461 1096 leader_election.cc:304] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters:
I20260204 08:53:10.114770 2329 raft_consensus.cc:2804] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260204 08:53:10.114823 1096 leader_election.cc:343] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4
I20260204 08:53:10.114907 2329 raft_consensus.cc:493] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1234acd7d0db405085a5d95795007134)
I20260204 08:53:10.115023 2329 raft_consensus.cc:3060] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [term 1 FOLLOWER]: Advancing to term 2
I20260204 08:53:10.116997 2329 raft_consensus.cc:515] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
I20260204 08:53:10.117600 2329 leader_election.cc:290] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 2 election: Requested vote from peers 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491), 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:53:10.118212 1019 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2de648ce42af4a2fbd494c8cc3e38bb4" candidate_uuid: "fdea56c2d1f5499087b939ec9f9dc070" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6d471268b9e643e5bbbf8bde1644a5ae"
I20260204 08:53:10.118433 1019 raft_consensus.cc:3060] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 1 FOLLOWER]: Advancing to term 2
W20260204 08:53:10.119077 1096 leader_election.cc:343] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4
I20260204 08:53:10.120621 1019 raft_consensus.cc:2468] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdea56c2d1f5499087b939ec9f9dc070 in term 2.
I20260204 08:53:10.121141 1096 leader_election.cc:304] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d471268b9e643e5bbbf8bde1644a5ae, fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134
I20260204 08:53:10.121480 2329 raft_consensus.cc:2804] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [term 2 FOLLOWER]: Leader election won for term 2
I20260204 08:53:10.121897 2329 raft_consensus.cc:697] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [term 2 LEADER]: Becoming Leader. State: Replica: fdea56c2d1f5499087b939ec9f9dc070, State: Running, Role: LEADER
I20260204 08:53:10.122273 2329 consensus_queue.cc:237] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } }
W20260204 08:53:10.227329 2610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260204 08:53:10.227679 2610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260204 08:53:10.227767 2610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260204 08:53:10.236454 2610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260204 08:53:10.236562 2610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260204 08:53:10.236620 2610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260204 08:53:10.236680 2610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260204 08:53:10.247766 2610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:42471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.104.190
--webserver_port=40787
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0efbebd487e7d0c7f21ab0d5a91e0fa70046af83
build type FASTDEBUG
built by None at 04 Feb 2026 08:43:20 UTC on 24a791456cd2
build id 10340
ASAN enabled
I20260204 08:53:10.249583 2610 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260204 08:53:10.251487 2610 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260204 08:53:10.260342 2618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:53:10.260344 2619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:53:10.260607 2621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260204 08:53:10.261441 2610 server_base.cc:1060] running on GCE node
I20260204 08:53:10.262234 2610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260204 08:53:10.263886 2610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260204 08:53:10.265107 2610 hybrid_clock.cc:648] HybridClock initialized: now 1770195190265036 us; error 62 us; skew 500 ppm
I20260204 08:53:10.265479 2610 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260204 08:53:10.267791 2610 webserver.cc:492] Webserver started at http://127.0.104.190:40787/ using document root <none> and password file <none>
I20260204 08:53:10.268774 2610 fs_manager.cc:362] Metadata directory not provided
I20260204 08:53:10.268954 2610 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260204 08:53:10.275377 2610 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260204 08:53:10.278718 2627 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260204 08:53:10.280354 2610 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260204 08:53:10.280541 2610 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data,/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34"
format_stamp: "Formatted at 2026-02-04 08:52:41 on dist-test-slave-vp4h"
I20260204 08:53:10.281303 2610 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260204 08:53:10.302575 2610 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260204 08:53:10.303491 2610 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260204 08:53:10.303767 2610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260204 08:53:10.323729 2610 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.190:42471
I20260204 08:53:10.323774 2678 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.190:42471 every 8 connection(s)
I20260204 08:53:10.325408 2610 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
I20260204 08:53:10.330626 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu as pid 2610
I20260204 08:53:10.334326 2679 sys_catalog.cc:263] Verifying existing consensus state
I20260204 08:53:10.339285 2679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Bootstrap starting.
I20260204 08:53:10.359716 2679 log.cc:826] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Log is configured to *not* fsync() on all Append() calls
I20260204 08:53:10.440707 2679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Bootstrap replayed 1/1 log segments. Stats: ops{read=110 overwritten=0 applied=110 ignored=0} inserts{seen=75 ignored=0} mutations{seen=94 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260204 08:53:10.441264 2679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Bootstrap complete.
I20260204 08:53:10.447006 2679 raft_consensus.cc:359] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:10.447762 2679 raft_consensus.cc:740] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebeeb5d4823b41b3a8e37546ff2f6e34, State: Initialized, Role: FOLLOWER
I20260204 08:53:10.448514 2679 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 110, Last appended: 7.110, Last appended by leader: 110, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:10.449447 2683 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:53:10.449909 2683 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:53:10.450740 2679 sys_catalog.cc:565] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: configured and running, proceeding with master startup.
I20260204 08:53:10.467773 2697 catalog_manager.cc:1269] Loaded cluster ID: 41c7773c62174f70835c9cfa744e1148
I20260204 08:53:10.467931 2697 catalog_manager.cc:1562] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: loading cluster ID for follower catalog manager: success
I20260204 08:53:10.475106 2697 catalog_manager.cc:1584] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: acquiring CA information for follower catalog manager: success
I20260204 08:53:10.487143 2697 catalog_manager.cc:1612] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260204 08:53:10.491874 2610 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:53:10.497428 2610 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:53:10.632257 1205 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:53:10.635726 2644 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" instance_seqno: 1770195163874581) as {username='slave'} at 127.0.104.130:33543; Asking this server to re-register.
I20260204 08:53:10.636687 1205 heartbeater.cc:461] Registering TS with master...
I20260204 08:53:10.639178 2644 ts_manager.cc:194] Registered new tserver with Master: fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:53:10.644918 1067 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:53:10.645346 1341 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:53:10.646593 2644 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" instance_seqno: 1770195163347042) as {username='slave'} at 127.0.104.129:54689; Asking this server to re-register.
I20260204 08:53:10.646950 2643 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1234acd7d0db405085a5d95795007134" instance_seqno: 1770195164422899) as {username='slave'} at 127.0.104.131:38847; Asking this server to re-register.
I20260204 08:53:10.647152 1067 heartbeater.cc:461] Registering TS with master...
I20260204 08:53:10.647523 1341 heartbeater.cc:461] Registering TS with master...
I20260204 08:53:10.649051 2644 ts_manager.cc:194] Registered new tserver with Master: 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:53:10.649242 2643 ts_manager.cc:194] Registered new tserver with Master: 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:53:10.997221 2706 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:10.997493 2706 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:10.999791 2706 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:11.000944 2628 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:53:11.002230 2630 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:53:11.002450 2628 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:11.002585 2630 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:11.002990 2706 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260204 08:53:11.030220 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 1: this message will repeat every 5th retry.
I20260204 08:53:11.373165 1019 raft_consensus.cc:1275] T 2de648ce42af4a2fbd494c8cc3e38bb4 P 6d471268b9e643e5bbbf8bde1644a5ae [term 2 FOLLOWER]: Refusing update from remote peer fdea56c2d1f5499087b939ec9f9dc070: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260204 08:53:11.373813 2329 consensus_queue.cc:1048] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260204 08:53:11.562732 2717 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:11.562927 2717 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:11.563635 2717 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:11.565764 2630 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:53:11.565968 2628 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:11.566102 2628 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:11.566401 2717 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:53:12.690678 2726 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:12.690901 2726 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:12.691612 2726 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:12.693924 2628 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:53:12.694103 2630 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:53:12.694232 2630 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:12.694465 2726 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:53:12.945456 418 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu pid 2610
I20260204 08:53:12.957255 2674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260204 08:53:13.071522 418 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu with pid 2610
I20260204 08:53:13.991653 418 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
/tmp/dist-test-taskfmwm4U/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:42471
--webserver_interface=127.0.104.190
--webserver_port=40787
--builtin_ntp_servers=127.0.104.148:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260204 08:53:14.369117 2747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260204 08:53:14.369583 2747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260204 08:53:14.369699 2747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260204 08:53:14.383885 2747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260204 08:53:14.384037 2747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260204 08:53:14.384119 2747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260204 08:53:14.384184 2747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260204 08:53:14.396998 2747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.104.148:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:41169
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.104.190:42471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.104.190
--webserver_port=40787
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0efbebd487e7d0c7f21ab0d5a91e0fa70046af83
build type FASTDEBUG
built by None at 04 Feb 2026 08:43:20 UTC on 24a791456cd2
build id 10340
ASAN enabled
I20260204 08:53:14.398993 2747 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260204 08:53:14.401217 2747 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260204 08:53:14.410938 2756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:53:14.410962 2757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260204 08:53:14.411577 2759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260204 08:53:14.412173 2747 server_base.cc:1060] running on GCE node
I20260204 08:53:14.413178 2747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260204 08:53:14.415380 2747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260204 08:53:14.416640 2747 hybrid_clock.cc:648] HybridClock initialized: now 1770195194416577 us; error 44 us; skew 500 ppm
I20260204 08:53:14.417117 2747 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260204 08:53:14.419829 2747 webserver.cc:492] Webserver started at http://127.0.104.190:40787/ using document root <none> and password file <none>
I20260204 08:53:14.420755 2747 fs_manager.cc:362] Metadata directory not provided
I20260204 08:53:14.420871 2747 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260204 08:53:14.427561 2747 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260204 08:53:14.431943 2765 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260204 08:53:14.434330 2747 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260204 08:53:14.434587 2747 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data,/tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34"
format_stamp: "Formatted at 2026-02-04 08:52:41 on dist-test-slave-vp4h"
I20260204 08:53:14.435606 2747 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260204 08:53:14.472576 2747 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260204 08:53:14.473665 2747 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260204 08:53:14.473991 2747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260204 08:53:14.495131 2747 rpc_server.cc:307] RPC server started. Bound to: 127.0.104.190:42471
I20260204 08:53:14.495177 2816 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.104.190:42471 every 8 connection(s)
I20260204 08:53:14.496953 2747 server_base.cc:1192] Dumped server information to /tmp/dist-test-taskfmwm4U/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1770195145154448-418-0/minicluster-data/master-0/data/info.pb
I20260204 08:53:14.503330 418 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskfmwm4U/build/asan/bin/kudu as pid 2747
I20260204 08:53:14.504769 2817 sys_catalog.cc:263] Verifying existing consensus state
I20260204 08:53:14.509337 2817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Bootstrap starting.
I20260204 08:53:14.531852 2817 log.cc:826] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Log is configured to *not* fsync() on all Append() calls
I20260204 08:53:14.617429 2817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Bootstrap replayed 1/1 log segments. Stats: ops{read=110 overwritten=0 applied=110 ignored=0} inserts{seen=75 ignored=0} mutations{seen=94 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260204 08:53:14.617978 2817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: Bootstrap complete.
I20260204 08:53:14.623870 2817 raft_consensus.cc:359] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:14.624697 2817 raft_consensus.cc:740] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebeeb5d4823b41b3a8e37546ff2f6e34, State: Initialized, Role: FOLLOWER
I20260204 08:53:14.625403 2817 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 110, Last appended: 7.110, Last appended by leader: 110, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:14.626286 2822 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } } }
I20260204 08:53:14.626566 2822 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: This master's current role is: FOLLOWER
I20260204 08:53:14.627408 2817 sys_catalog.cc:565] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [sys.catalog]: configured and running, proceeding with master startup.
I20260204 08:53:14.640095 2836 catalog_manager.cc:1269] Loaded cluster ID: 41c7773c62174f70835c9cfa744e1148
I20260204 08:53:14.640213 2836 catalog_manager.cc:1562] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: loading cluster ID for follower catalog manager: success
I20260204 08:53:14.645365 2836 catalog_manager.cc:1584] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: acquiring CA information for follower catalog manager: success
I20260204 08:53:14.650278 2836 catalog_manager.cc:1612] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260204 08:53:14.661721 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:53:14.666066 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:53:14.672442 1341 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:53:14.676102 2782 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1234acd7d0db405085a5d95795007134" instance_seqno: 1770195164422899) as {username='slave'} at 127.0.104.131:33787; Asking this server to re-register.
I20260204 08:53:14.677079 1341 heartbeater.cc:461] Registering TS with master...
I20260204 08:53:14.679467 2782 ts_manager.cc:194] Registered new tserver with Master: 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
I20260204 08:53:14.710111 1205 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:53:14.712226 2782 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" instance_seqno: 1770195163874581) as {username='slave'} at 127.0.104.130:35173; Asking this server to re-register.
I20260204 08:53:14.713089 1205 heartbeater.cc:461] Registering TS with master...
I20260204 08:53:14.715744 2781 ts_manager.cc:194] Registered new tserver with Master: fdea56c2d1f5499087b939ec9f9dc070 (127.0.104.130:33737)
I20260204 08:53:14.901638 1067 heartbeater.cc:344] Connected to a master server at 127.0.104.190:42471
I20260204 08:53:14.903604 2781 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" instance_seqno: 1770195163347042) as {username='slave'} at 127.0.104.129:50287; Asking this server to re-register.
I20260204 08:53:14.904418 1067 heartbeater.cc:461] Registering TS with master...
I20260204 08:53:14.906368 2781 ts_manager.cc:194] Registered new tserver with Master: 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675)
I20260204 08:53:15.220599 2844 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:15.220885 2844 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:15.222800 2844 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:15.225157 2766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:53:15.226504 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:53:15.226756 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:53:15.226867 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:15.227268 2844 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:53:15.932523 2853 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:15.932736 2853 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:15.933512 2853 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:15.936097 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:53:15.936358 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:53:15.936511 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:15.936784 2853 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:53:16.624050 2860 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:16.624276 2860 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:53:16.625176 2860 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:53:16.626139 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:53:16.626367 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:53:16.626462 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:53:16.626725 2859 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:53:16.835770 2866 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:16.835996 2866 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:16.836735 2866 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:16.838785 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:53:16.838951 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:16.839111 2766 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:16.839371 2866 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:53:16.866415 418 master-stress-test.cc:577] Killing test thread 1/25
I20260204 08:53:17.346975 2873 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:17.347180 2873 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:17.347889 2873 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:17.349974 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:53:17.350147 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:53:17.350297 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:17.350591 2873 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:53:18.363226 2888 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:18.363451 2888 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:18.364123 2888 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:18.369150 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:53:18.369418 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:18.369529 2766 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:18.369809 2888 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:53:19.224057 2903 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:19.224257 2903 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:19.224879 2903 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:19.227331 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:53:19.227538 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:19.227640 2766 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:19.227905 2903 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:53:19.669400 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:53:19.669637 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:53:19.910595 2902 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:19.910791 2902 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:53:19.911448 2902 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:53:19.912411 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:53:19.912629 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:53:19.912737 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:53:19.912999 2902 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260204 08:53:21.098932 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 6: this message will repeat every 5th retry.
I20260204 08:53:23.697412 2952 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:23.697633 2952 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:23.698381 2952 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:23.699429 2768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20260204 08:53:23.700968 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:53:23.701191 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:53:23.701301 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:23.701558 2952 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:53:24.672777 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:53:24.672976 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:53:29.676267 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:53:29.676551 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:53:30.706122 3026 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:30.706363 3026 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:30.707170 3026 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:30.708334 2768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:53:30.709728 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:53:30.709903 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:30.710045 2766 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:30.710350 3026 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:53:31.123450 3011 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:31.123647 3011 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:53:31.124404 3011 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:53:31.125435 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:53:31.125674 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:53:31.125787 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:53:31.126096 3011 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260204 08:53:31.598042 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 11: this message will repeat every 5th retry.
I20260204 08:53:34.679703 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:53:34.679914 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:53:36.867043 3094 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:36.867264 3094 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:36.867975 3094 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:36.868927 2768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:53:36.870455 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:53:36.870635 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:36.870774 2766 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:36.871045 3094 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:53:37.633414 3107 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:37.633607 3107 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:37.634469 3107 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:37.636454 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:53:37.636632 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:37.636775 2766 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:37.637072 3107 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:53:39.688658 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:53:39.688855 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260204 08:53:42.187783 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 16: this message will repeat every 5th retry.
I20260204 08:53:44.692669 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:53:44.692920 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260204 08:53:44.924052 1206 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:38981 (153 consecutive failures): Network error: Failed to ping master at 127.0.104.188:38981: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:53:44.987963 1068 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:38981 (134 consecutive failures): Network error: Failed to ping master at 127.0.104.188:38981: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:53:45.077220 1342 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:38981 (132 consecutive failures): Network error: Failed to ping master at 127.0.104.188:38981: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:47.145897 3198 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:47.146142 3198 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:47.146894 3198 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:47.153282 2766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260204 08:53:47.154791 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:53:47.154980 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:53:47.155155 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:47.155447 3198 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:53:49.695945 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:53:49.696149 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:53:49.784257 3207 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:49.784507 3207 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:53:49.785341 3207 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:53:49.786325 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:53:49.786576 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:53:49.786670 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:53:49.786967 3207 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260204 08:53:52.408026 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 21: this message will repeat every 5th retry.
I20260204 08:53:54.308148 3269 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:54.308398 3269 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:53:54.309103 3269 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:53:54.310057 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:53:54.310329 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:53:54.310447 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:53:54.310719 3269 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:53:54.699018 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:53:54.699224 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:53:57.936882 3315 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:53:57.937104 3315 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:53:57.937887 3315 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:53:57.939134 2768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:53:57.940564 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:53:57.940809 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:53:57.940963 2766 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:53:57.941308 3315 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:53:59.702453 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:53:59.702641 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260204 08:54:03.372001 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 26: this message will repeat every 5th retry.
I20260204 08:54:04.705574 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:54:04.705749 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:54:07.735806 3418 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:54:07.736040 3418 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:54:07.736935 3418 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:54:07.737959 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:54:07.738200 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:54:07.738301 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:54:07.738571 3418 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:54:09.709048 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:54:09.709256 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:54:12.856320 3472 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:54:12.856537 3472 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:54:12.857255 3472 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:54:12.858479 2766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:54:12.860433 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:54:12.860646 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:54:12.860783 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:54:12.861018 3472 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260204 08:54:13.868904 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 31: this message will repeat every 5th retry.
I20260204 08:54:14.633948 3494 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:54:14.634151 3494 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:54:14.634945 3494 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:54:14.637529 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:54:14.637859 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:54:14.638016 2766 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:54:14.638286 3494 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:54:14.712368 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:54:14.712559 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:54:19.715538 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:54:19.715731 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:54:20.165750 3542 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:54:20.165961 3542 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:54:20.166757 3542 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:54:20.167742 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:54:20.167963 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:54:20.168041 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:54:20.168282 3542 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260204 08:54:23.373582 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 36: this message will repeat every 5th retry.
I20260204 08:54:24.719161 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:54:24.719372 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:54:26.412420 3611 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:54:26.412638 3611 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:54:26.413352 3611 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:54:26.414394 2766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260204 08:54:26.416003 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:54:26.416195 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:54:26.416368 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:54:26.416769 3611 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:54:28.989971 3639 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:54:28.990177 3639 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:54:28.990940 3639 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:54:28.991910 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:54:28.992187 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:54:28.992286 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:54:28.992568 3639 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:54:29.723073 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:54:29.723285 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260204 08:54:33.332815 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 41: this message will repeat every 5th retry.
I20260204 08:54:34.726255 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:54:34.726438 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:54:39.729503 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:54:39.729689 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:54:40.846979 3769 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:54:40.847174 3769 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:54:40.847848 3769 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:54:40.848932 2768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:54:40.850093 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:54:40.850255 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:54:40.850405 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:54:40.850704 3769 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:54:41.014699 3764 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:54:41.014943 3764 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:54:41.015618 3764 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:54:41.016616 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:54:41.016959 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:54:41.017064 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:54:41.017304 3764 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260204 08:54:43.232231 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 46: this message will repeat every 5th retry.
I20260204 08:54:44.732434 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:54:44.732613 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260204 08:54:45.170495 1069 heartbeater.cc:646] Failed to heartbeat to 127.0.104.189:37005 (195 consecutive failures): Network error: Failed to ping master at 127.0.104.189:37005: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:54:45.228874 1206 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:38981 (303 consecutive failures): Network error: Failed to ping master at 127.0.104.188:38981: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:54:45.229115 1342 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:38981 (252 consecutive failures): Network error: Failed to ping master at 127.0.104.188:38981: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:54:47.134228 3817 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:54:47.134456 3817 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:54:47.135329 3817 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:54:47.136232 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:54:47.136507 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:54:47.136598 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:54:47.136881 3817 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:54:49.735672 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:54:49.735872 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:54:51.898692 3884 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:54:51.898885 3884 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:54:51.899560 3884 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:54:51.900542 2766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:54:51.902166 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:54:51.902333 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:54:51.902498 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:54:51.902769 3884 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260204 08:54:53.369524 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 51: this message will repeat every 5th retry.
I20260204 08:54:54.739295 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:54:54.739475 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:54:56.840327 3931 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:54:56.840538 3931 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:54:56.841228 3931 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:54:56.842096 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:54:56.842307 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:54:56.842386 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:54:56.842643 3931 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:54:59.744656 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:54:59.744835 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:55:03.167142 3980 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:55:03.167348 3980 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:55:03.168161 3980 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:55:03.168987 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:55:03.169162 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:55:03.169245 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:55:03.169468 3980 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260204 08:55:03.211493 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 56: this message will repeat every 5th retry.
I20260204 08:55:04.154153 4008 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:55:04.154361 4008 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:55:04.155184 4008 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:55:04.156162 2768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:55:04.157444 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:55:04.157636 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:55:04.157764 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:55:04.158007 4008 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:55:04.747483 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:55:04.747655 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:55:06.588424 4028 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:55:06.588614 4028 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:55:06.589232 4028 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:55:06.590312 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:55:06.591558 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:55:06.591688 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:55:06.592018 4038 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:55:09.750483 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:55:09.750679 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:55:13.175729 4102 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:55:13.175963 4102 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:55:13.176757 4102 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:55:13.177845 2766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:55:13.179418 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:55:13.179621 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:55:13.179724 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:55:13.179988 4102 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260204 08:55:13.377336 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 61: this message will repeat every 5th retry.
I20260204 08:55:14.657536 2771 maintenance_manager.cc:419] P ebeeb5d4823b41b3a8e37546ff2f6e34: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033342
I20260204 08:55:14.722215 2770 maintenance_manager.cc:643] P ebeeb5d4823b41b3a8e37546ff2f6e34: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.064s user 0.029s sys 0.004s Metrics: {"bytes_written":24608,"cfile_init":1,"dirs.queue_time_us":468,"dirs.run_cpu_time_us":228,"dirs.run_wall_time_us":814,"drs_written":1,"lbm_read_time_us":75,"lbm_reads_lt_1ms":4,"lbm_write_time_us":748,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":68,"thread_start_us":542,"threads_started":2,"wal-append.queue_time_us":350}
I20260204 08:55:14.723611 2771 maintenance_manager.cc:419] P ebeeb5d4823b41b3a8e37546ff2f6e34: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4247 bytes on disk
I20260204 08:55:14.724982 2770 maintenance_manager.cc:643] P ebeeb5d4823b41b3a8e37546ff2f6e34: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":89,"lbm_reads_lt_1ms":4}
I20260204 08:55:14.753635 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:55:14.753827 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:55:19.579499 4192 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:55:19.579715 4192 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:55:19.580639 4192 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:55:19.581418 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:55:19.581609 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:55:19.581705 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:55:19.581967 4192 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:55:19.756986 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:55:19.757184 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260204 08:55:23.258270 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 66: this message will repeat every 5th retry.
I20260204 08:55:24.760015 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:55:24.760200 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:55:27.180001 4302 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:55:27.180245 4302 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:55:27.181059 4302 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:55:27.185205 2768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:55:27.186856 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:55:27.187049 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:55:27.187184 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:55:27.187484 4302 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:55:29.763343 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:55:29.763561 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260204 08:55:32.983529 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 71: this message will repeat every 5th retry.
I20260204 08:55:34.359010 4395 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:55:34.359234 4395 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:55:34.359925 4395 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:55:34.360821 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:55:34.361066 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:55:34.361157 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:55:34.361387 4395 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:55:34.766520 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:55:34.766697 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:55:39.394694 4473 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:55:39.395004 4473 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:55:39.395921 4473 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:55:39.396827 2768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:55:39.398262 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:55:39.398473 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:55:39.398588 2766 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:55:39.398871 4473 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:55:39.769200 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:55:39.769413 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260204 08:55:42.674222 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 76: this message will repeat every 5th retry.
I20260204 08:55:44.772527 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:55:44.772729 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260204 08:55:45.349936 1069 heartbeater.cc:646] Failed to heartbeat to 127.0.104.189:37005 (315 consecutive failures): Network error: Failed to ping master at 127.0.104.189:37005: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:55:45.374238 1342 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:38981 (372 consecutive failures): Network error: Failed to ping master at 127.0.104.188:38981: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:55:45.429672 1206 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:38981 (452 consecutive failures): Network error: Failed to ping master at 127.0.104.188:38981: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:55:49.775563 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:55:49.775743 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:55:52.061293 4647 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:55:52.061501 4647 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:55:52.062335 4647 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:55:52.063400 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:55:52.063616 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:55:52.063704 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:55:52.063943 4647 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260204 08:55:52.854962 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 81: this message will repeat every 5th retry.
I20260204 08:55:54.708199 4682 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:55:54.708519 4682 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:55:54.709561 4682 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:55:54.710526 2768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:55:54.711774 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:55:54.711989 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:55:54.712152 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:55:54.712486 4682 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:55:54.778683 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:55:54.778905 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:55:59.781935 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:55:59.782145 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:56:00.479128 4754 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:56:00.479328 4754 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:56:00.479965 4754 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:56:00.480984 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:56:00.481206 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:56:00.481326 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:56:00.481779 4754 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260204 08:56:03.198160 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 86: this message will repeat every 5th retry.
I20260204 08:56:04.230319 4814 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:56:04.230563 4814 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:56:04.231271 4814 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:56:04.232230 2768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:56:04.233376 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:56:04.233578 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:56:04.233709 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:56:04.233966 4814 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:56:04.785079 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:56:04.785286 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:56:09.235767 4863 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:56:09.235984 4863 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:56:09.236828 4863 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:56:09.237779 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:56:09.238010 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:56:09.238098 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:56:09.238349 4863 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:56:09.787948 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:56:09.788152 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:56:11.546396 4917 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:56:11.546584 4917 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:56:11.547250 4917 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:56:11.548115 2768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:56:11.549652 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:56:11.549856 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:56:11.550001 2766 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:56:11.550326 4917 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260204 08:56:13.451362 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 91: this message will repeat every 5th retry.
I20260204 08:56:14.791317 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:56:14.791512 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:56:17.605583 5003 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:56:17.605791 5003 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:56:17.606513 5003 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:56:17.608179 2768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:56:17.609781 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:56:17.609977 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:56:17.610139 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:56:17.610450 5003 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:56:19.794776 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:56:19.794972 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:56:21.350517 5054 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:56:21.350761 5054 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:56:21.351527 5054 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:56:21.354310 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:56:21.354543 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:56:21.354756 2766 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:56:21.355069 5054 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260204 08:56:23.201536 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 96: this message will repeat every 5th retry.
I20260204 08:56:24.798146 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:56:24.798322 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:56:26.099166 5119 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:56:26.099437 5119 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:56:26.100349 5119 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:56:26.101394 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:56:26.101599 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:56:26.101714 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:56:26.101986 5119 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:56:29.801406 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:56:29.801602 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260204 08:56:33.245033 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 101: this message will repeat every 5th retry.
I20260204 08:56:34.804657 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:56:34.804853 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:56:37.719424 5274 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:56:37.719631 5274 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:56:37.720546 5274 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:56:37.721578 2766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260204 08:56:37.722919 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:56:37.723142 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:56:37.723255 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:56:37.723521 5274 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:56:39.808041 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:56:39.808225 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260204 08:56:42.986830 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 106: this message will repeat every 5th retry.
I20260204 08:56:43.988531 5334 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:56:43.988758 5334 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:56:43.990046 5334 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:56:43.991001 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:56:43.991189 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:56:43.991281 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:56:43.991535 5334 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:56:44.246098 5364 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:56:44.246335 5364 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:56:44.247095 5364 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:56:44.247973 2768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:56:44.249651 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:56:44.249818 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:56:44.249934 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:56:44.250176 5364 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:56:44.811229 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:56:44.811455 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260204 08:56:45.526647 1342 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:38981 (492 consecutive failures): Network error: Failed to ping master at 127.0.104.188:38981: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:56:45.536522 1069 heartbeater.cc:646] Failed to heartbeat to 127.0.104.189:37005 (435 consecutive failures): Network error: Failed to ping master at 127.0.104.189:37005: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:56:45.923324 1206 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:38981 (603 consecutive failures): Network error: Failed to ping master at 127.0.104.188:38981: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:56:49.814468 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:56:49.814679 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:56:51.433362 5445 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:56:51.433549 5445 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:56:51.434207 5445 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:56:51.435094 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:56:51.435281 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:56:51.435357 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:56:51.435660 5445 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260204 08:56:52.689790 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 111: this message will repeat every 5th retry.
I20260204 08:56:54.817598 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:56:54.817795 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:56:56.385628 5529 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:56:56.385864 5529 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:56:56.386686 5529 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:56:56.387706 2768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:56:56.389125 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:56:56.389309 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:56:56.389433 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:56:56.389700 5529 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:56:57.807627 5555 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:56:57.807816 5555 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:56:57.808516 5555 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:56:57.810588 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:56:57.810773 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:56:57.810905 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:56:57.811154 5555 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:56:58.720474 5552 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:56:58.720670 5552 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:56:58.721342 5552 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:56:58.722330 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:56:58.722501 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:56:58.722560 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:56:58.722770 5552 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:56:59.821059 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:56:59.821275 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:57:02.367699 5608 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:57:02.367878 5608 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:57:02.368656 5608 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:57:02.369732 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:57:02.369936 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:57:02.370011 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:57:02.370311 5608 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260204 08:57:03.363574 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 116: this message will repeat every 5th retry.
I20260204 08:57:04.824024 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:57:04.824223 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:57:09.827257 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:57:09.827454 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:57:12.237108 5744 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:57:12.237326 5744 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:57:12.238063 5744 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:57:12.239786 2766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260204 08:57:12.241242 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:57:12.241439 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:57:12.241590 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:57:12.241904 5744 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260204 08:57:13.304224 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 121: this message will repeat every 5th retry.
I20260204 08:57:14.830333 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:57:14.830529 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:57:19.833294 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:57:19.833489 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:57:21.731168 5850 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:57:21.731369 5850 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:57:21.732052 5850 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:57:21.733055 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:57:21.733249 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:57:21.733330 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:57:21.733556 5850 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260204 08:57:23.171432 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 126: this message will repeat every 5th retry.
I20260204 08:57:24.836203 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:57:24.836427 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:57:26.838327 5936 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:57:26.838559 5936 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:57:26.839293 5936 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:57:26.840229 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:57:26.840466 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:57:26.840544 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:57:26.840797 5936 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:57:29.839831 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:57:29.840040 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:57:30.158596 5990 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:57:30.158859 5990 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:57:30.159744 5990 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:57:30.160491 2768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:57:30.161886 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:57:30.162065 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:57:30.162182 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:57:30.162441 5990 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260204 08:57:33.086400 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 131: this message will repeat every 5th retry.
I20260204 08:57:34.843078 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:57:34.843287 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:57:36.537535 6067 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:57:36.537824 6067 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:57:36.538573 6067 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:57:36.539675 2766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:57:36.541235 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:57:36.541414 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
I20260204 08:57:36.541543 2766 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:57:36.541834 6067 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260204 08:57:39.846411 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:57:39.846603 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260204 08:57:41.608605 6136 raft_consensus.cc:493] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:57:41.608831 6136 raft_consensus.cc:515] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebeeb5d4823b41b3a8e37546ff2f6e34" member_type: VOTER last_known_addr { host: "127.0.104.190" port: 42471 } } peers { permanent_uuid: "d083ecc9895345e5b06cd00ef0246c71" member_type: VOTER last_known_addr { host: "127.0.104.189" port: 37005 } } peers { permanent_uuid: "c1e719e457e94a5da71929720411ae69" member_type: VOTER last_known_addr { host: "127.0.104.188" port: 38981 } }
I20260204 08:57:41.609545 6136 leader_election.cc:290] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005), c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981)
W20260204 08:57:41.610512 2766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260204 08:57:41.611907 2766 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer c1e719e457e94a5da71929720411ae69 (127.0.104.188:38981): Network error: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:57:41.612071 2768 leader_election.cc:336] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d083ecc9895345e5b06cd00ef0246c71 (127.0.104.189:37005): Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
I20260204 08:57:41.612191 2768 leader_election.cc:304] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ebeeb5d4823b41b3a8e37546ff2f6e34; no voters: c1e719e457e94a5da71929720411ae69, d083ecc9895345e5b06cd00ef0246c71
I20260204 08:57:41.612459 6136 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ebeeb5d4823b41b3a8e37546ff2f6e34 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260204 08:57:42.907851 1096 consensus_peers.cc:597] T 2de648ce42af4a2fbd494c8cc3e38bb4 P fdea56c2d1f5499087b939ec9f9dc070 -> Peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Couldn't send request to peer 1234acd7d0db405085a5d95795007134. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2de648ce42af4a2fbd494c8cc3e38bb4. This is attempt 136: this message will repeat every 5th retry.
I20260204 08:57:43.124361 6157 raft_consensus.cc:493] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260204 08:57:43.124549 6157 raft_consensus.cc:515] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdea56c2d1f5499087b939ec9f9dc070" member_type: VOTER last_known_addr { host: "127.0.104.130" port: 33737 } } peers { permanent_uuid: "6d471268b9e643e5bbbf8bde1644a5ae" member_type: VOTER last_known_addr { host: "127.0.104.129" port: 36675 } } peers { permanent_uuid: "1234acd7d0db405085a5d95795007134" member_type: VOTER last_known_addr { host: "127.0.104.131" port: 46491 } }
I20260204 08:57:43.125317 6157 leader_election.cc:290] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675), 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491)
W20260204 08:57:43.126262 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d471268b9e643e5bbbf8bde1644a5ae (127.0.104.129:36675): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
W20260204 08:57:43.126441 1096 leader_election.cc:343] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1234acd7d0db405085a5d95795007134 (127.0.104.131:46491): Not found: Tablet not found: 742c8a8c99464661873f292406ac5027
I20260204 08:57:43.126506 1096 leader_election.cc:304] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fdea56c2d1f5499087b939ec9f9dc070; no voters: 1234acd7d0db405085a5d95795007134, 6d471268b9e643e5bbbf8bde1644a5ae
I20260204 08:57:43.126735 6157 raft_consensus.cc:2749] T 742c8a8c99464661873f292406ac5027 P fdea56c2d1f5499087b939ec9f9dc070 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260204 08:57:44.849629 2747 master_runner.cc:186] Error getting master registration for 127.0.104.189:37005: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111),
I20260204 08:57:44.849810 2747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260204 08:57:45.672891 1342 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:38981 (612 consecutive failures): Network error: Failed to ping master at 127.0.104.188:38981: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:57:45.721657 1069 heartbeater.cc:646] Failed to heartbeat to 127.0.104.189:37005 (555 consecutive failures): Network error: Failed to ping master at 127.0.104.189:37005: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:57:46.354511 1206 heartbeater.cc:646] Failed to heartbeat to 127.0.104.188:38981 (755 consecutive failures): Network error: Failed to ping master at 127.0.104.188:38981: Client connection negotiation failed: client connection to 127.0.104.188:38981: connect: Connection refused (error 111)
W20260204 08:57:48.463274 1353 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:57:48.463554 1353 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:57:48.463687 1353 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
W20260204 08:57:48.463896 1353 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
F20260204 08:57:48.474860 1374 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260204 08:57:48.475401 1360 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_27cce8dee4984465be213744abfadc9f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
*** Check failure stack trace: ***
*** Aborted at 1770195468 (unix time) try "date -d @1770195468" if you are using GNU date ***
F20260204 08:57:48.475401 1360 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_27cce8dee4984465be213744abfadc9f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.476758 1378 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
*** Check failure stack trace: ***
W20260204 08:57:48.478812 1353 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.104.189:37005)
W20260204 08:57:48.479143 1353 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.104.189:37005)
F20260204 08:57:48.475401 1360 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_27cce8dee4984465be213744abfadc9f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.476758 1378 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.481895 1366 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c2dfe1ff212d4d2b8d46f11cb24ac67e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260204 08:57:48.475401 1360 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_27cce8dee4984465be213744abfadc9f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.476758 1378 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.481895 1366 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c2dfe1ff212d4d2b8d46f11cb24ac67e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.482676 1369 master-stress-test.cc:328] Check failed: _s.ok() Bad status: F20260204 08:57:48.482704 Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)1371 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260204 08:57:48.475401 1360 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_27cce8dee4984465be213744abfadc9f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.476758 1378 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.481895 1366 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c2dfe1ff212d4d2b8d46f11cb24ac67e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.482676 1369 master-stress-test.cc:328] Check failed: _s.ok() Bad status: F20260204 08:57:48.482704 Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)1371 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.483014 1364 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_85f07b4907b24f1ca63791592455b59c on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260204 08:57:48.475401 1360 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_27cce8dee4984465be213744abfadc9f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.476758 1378 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.481895 1366 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c2dfe1ff212d4d2b8d46f11cb24ac67e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.482676 1369 master-stress-test.cc:328] Check failed: _s.ok() Bad status: F20260204 08:57:48.482704 Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)1371 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.483014 1364 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_85f07b4907b24f1ca63791592455b59c on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
W20260204 08:57:48.483300 1353 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.104.189:37005)
*** SIGABRT (@0x3e8000001a2) received by PID 418 (TID 0x7f59f2223700) from PID 418; stack trace: ***
F20260204 08:57:48.475401 1360 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_27cce8dee4984465be213744abfadc9f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.476758 1378 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.481895 1366 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c2dfe1ff212d4d2b8d46f11cb24ac67e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.482676 1369 master-stress-test.cc:328] Check failed: _s.ok() Bad status: F20260204 08:57:48.482704 Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)1371 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.483014 1364 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_85f07b4907b24f1ca63791592455b59c on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.483902 1372 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260204 08:57:48.475401 1360 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_27cce8dee4984465be213744abfadc9f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.476758 1378 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.481895 1366 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c2dfe1ff212d4d2b8d46f11cb24ac67e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.482676 1369 master-stress-test.cc:328] Check failed: _s.ok() Bad status: F20260204 08:57:48.482704 Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)1371 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.483014 1364 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_85f07b4907b24f1ca63791592455b59c on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.483902 1372 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.484319 1370 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260204 08:57:48.475401 1360 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_27cce8dee4984465be213744abfadc9f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.476758 1378 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.481895 1366 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_c2dfe1ff212d4d2b8d46f11cb24ac67e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.482676 1369 master-stress-test.cc:328] Check failed: _s.ok() Bad status: F20260204 08:57:48.482704 Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)1371 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.483014 1364 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_85f07b4907b24f1ca63791592455b59c on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.483902 1372 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.484319 1370 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)F20260204 08:57:48.485257 1373 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.104.190:42471,127.0.104.189:37005,127.0.104.188:38981, num_attempts: 745) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.104.189:37005: connect: Connection refused (error 111)
*** Check failure stack trace: ***
@ 0x7f5a0c5ff980 (unknown) at ??:0
@ 0x7f5a053d2fb7 gsignal at ??:0
@ 0x7f5a053d4921 abort at ??:0
@ 0x7f5a0810adcd google::LogMessage::Fail() at ??:0
@ 0x7f5a0810eb93 google::LogMessage::SendToLog() at ??:0
@ 0x7f5a0810a7cc google::LogMessage::Flush() at ??:0
@ 0x7f5a0810bf59 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x40cb7c kudu::MasterStressTest::DeleteTableThread() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:351
@ 0x44aaf4 std::__invoke_impl<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:73
@ 0x44a9ad std::__invoke<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:95
@ 0x44a96a _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:234
@ 0x44a91e std::thread::_Invoker<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:243
@ 0x44a659 std::thread::_State_impl<>::_M_run() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:186
@ 0x7f5a05a5a6df (unknown) at ??:0
@ 0x7f5a0c5f46db start_thread at ??:0
@ 0x7f5a054b571f clone at ??:0