Note: This is test shard 3 of 3.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/2
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260806 02:45:02.033574 20720 test_util.cc:268] Using random seed: -538596748
2026-08-06T02:45:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:45:02Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskYdWHu7/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskYdWHu7/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/security.properties
2026-08-06 02:45:08: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskYdWHu7/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskYdWHu7/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260806 02:45:16.706354 20720 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
/tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.62:33227
--webserver_interface=127.20.60.62
--webserver_port=0
--builtin_ntp_servers=127.20.60.20:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260806 02:45:17.144068 21045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:45:17.145076 21045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:45:17.145252 21045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:45:17.152721 21045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:45:17.152873 21045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:45:17.152981 21045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:45:17.153087 21045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:45:17.163679 21045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.60.20:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.62:33227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.60.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 01:55:54 UTC on ccycloud.quasar-csdhqu.root.comops.site
build id 4569536
ASAN enabled
I20260806 02:45:17.166791 21045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:45:17.170607 21045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:45:17.188884 21051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:17.188844 21052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:17.188707 21054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:45:17.189775 21045 server_base.cc:1034] running on GCE node
I20260806 02:45:17.192323 21045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:45:17.194697 21045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:45:17.195971 21045 hybrid_clock.cc:648] HybridClock initialized: now 1785984317195861 us; error 71 us; skew 500 ppm
I20260806 02:45:17.197443 21045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 02:45:17.200629 21045 webserver.cc:460] Webserver started at http://127.20.60.62:34225/ using document root <none> and password file <none>
I20260806 02:45:17.202042 21045 fs_manager.cc:359] Metadata directory not provided
I20260806 02:45:17.202242 21045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:45:17.202823 21045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:45:17.208168 21045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data/instance:
uuid: "a225e002d607451a94b3686f10aeea9a"
format_stamp: "Formatted at 2026-08-06 02:45:17 on dist-test-slave-bw88"
I20260806 02:45:17.209232 21045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal/instance:
uuid: "a225e002d607451a94b3686f10aeea9a"
format_stamp: "Formatted at 2026-08-06 02:45:17 on dist-test-slave-bw88"
I20260806 02:45:17.219664 21045 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.005s sys 0.002s
I20260806 02:45:17.225960 21063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:17.227957 21045 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260806 02:45:17.228207 21045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data,/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
uuid: "a225e002d607451a94b3686f10aeea9a"
format_stamp: "Formatted at 2026-08-06 02:45:17 on dist-test-slave-bw88"
I20260806 02:45:17.228452 21045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:45:17.257184 21045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:45:17.258283 21045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:45:17.258651 21045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:45:17.285712 21045 rpc_server.cc:307] RPC server started. Bound to: 127.20.60.62:33227
I20260806 02:45:17.285722 21128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.60.62:33227 every 8 connection(s)
I20260806 02:45:17.287901 21045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data/info.pb
I20260806 02:45:17.298168 20720 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu as pid 21045
I20260806 02:45:17.298453 20720 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal/instance
I20260806 02:45:17.299526 21129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:17.301434 20720 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
/tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.61:36797
--webserver_interface=127.20.60.61
--webserver_port=0
--builtin_ntp_servers=127.20.60.20:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260806 02:45:17.313264 21129 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:45:17.347816 21129 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } has no permanent_uuid. Determining permanent_uuid...
W20260806 02:45:17.349259 21064 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:45:17.352061 21129 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.60.61:36797: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:45:17.401485 21129 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } attempt: 1
W20260806 02:45:17.403609 21129 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.60.61:36797: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:45:17.471884 21129 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } attempt: 2
W20260806 02:45:17.474311 21129 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.60.61:36797: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:45:17.565616 21129 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } attempt: 3
W20260806 02:45:17.567986 21129 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.60.61:36797: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:45:17.657047 21132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:45:17.657410 21132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:45:17.657558 21132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:45:17.664856 21132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:45:17.665014 21132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:45:17.665118 21132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:45:17.665225 21132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:45:17.674836 21132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.60.20:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.61:36797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.60.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 01:55:54 UTC on ccycloud.quasar-csdhqu.root.comops.site
build id 4569536
ASAN enabled
I20260806 02:45:17.676335 21132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:45:17.678318 21132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:45:17.691762 21146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:17.691874 21145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:17.691855 21148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:45:17.692837 21132 server_base.cc:1034] running on GCE node
I20260806 02:45:17.693701 21132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:45:17.695127 21132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:45:17.696358 21132 hybrid_clock.cc:648] HybridClock initialized: now 1785984317696296 us; error 34 us; skew 500 ppm
I20260806 02:45:17.696709 21132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 02:45:17.699223 21132 webserver.cc:460] Webserver started at http://127.20.60.61:46303/ using document root <none> and password file <none>
I20260806 02:45:17.699939 21132 fs_manager.cc:359] Metadata directory not provided
I20260806 02:45:17.700045 21132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:45:17.700306 21132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:45:17.703270 21132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data/instance:
uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22"
format_stamp: "Formatted at 2026-08-06 02:45:17 on dist-test-slave-bw88"
I20260806 02:45:17.704025 21132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal/instance:
uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22"
format_stamp: "Formatted at 2026-08-06 02:45:17 on dist-test-slave-bw88"
I20260806 02:45:17.709496 21132 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 02:45:17.711227 21129 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } attempt: 4
I20260806 02:45:17.713300 21156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
W20260806 02:45:17.713459 21129 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.60.61:36797: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:45:17.714819 21132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 02:45:17.714974 21132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data,/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22"
format_stamp: "Formatted at 2026-08-06 02:45:17 on dist-test-slave-bw88"
I20260806 02:45:17.715137 21132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:45:17.732581 21132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:45:17.733454 21132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:45:17.733778 21132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:45:17.754326 21132 rpc_server.cc:307] RPC server started. Bound to: 127.20.60.61:36797
I20260806 02:45:17.754374 21225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.60.61:36797 every 8 connection(s)
I20260806 02:45:17.755820 21132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data/info.pb
I20260806 02:45:17.760043 21226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:17.764243 20720 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu as pid 21132
I20260806 02:45:17.764424 20720 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal/instance
I20260806 02:45:17.766049 20720 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
/tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.60:42309
--webserver_interface=127.20.60.60
--webserver_port=0
--builtin_ntp_servers=127.20.60.20:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260806 02:45:17.768157 21226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:45:17.785037 21226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:45:17.795504 21226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } has no permanent_uuid. Determining permanent_uuid...
W20260806 02:45:17.796646 21163 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
W20260806 02:45:17.798424 21226 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.60.60:42309: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:17.847769 21226 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } attempt: 1
W20260806 02:45:17.850039 21226 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.60.60:42309: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:17.918289 21226 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } attempt: 2
W20260806 02:45:17.920595 21226 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.60.60:42309: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:18.011967 21226 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } attempt: 3
I20260806 02:45:18.012807 21129 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } attempt: 5
W20260806 02:45:18.015005 21226 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.60.60:42309: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:18.024977 21129 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } has no permanent_uuid. Determining permanent_uuid...
W20260806 02:45:18.027522 21129 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.60.60:42309: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:18.078840 21129 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } attempt: 1
W20260806 02:45:18.081423 21129 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.60.60:42309: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:18.149799 21129 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } attempt: 2
W20260806 02:45:18.152622 21129 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.60.60:42309: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:18.158363 21226 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } attempt: 4
W20260806 02:45:18.161007 21226 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.60.60:42309: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
W20260806 02:45:18.207216 21228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:45:18.207579 21228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:45:18.207793 21228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:45:18.216428 21228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:45:18.216619 21228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:45:18.216737 21228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:45:18.216863 21228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:45:18.228487 21228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.60.20:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.60:42309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.60.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 01:55:54 UTC on ccycloud.quasar-csdhqu.root.comops.site
build id 4569536
ASAN enabled
I20260806 02:45:18.230113 21228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:45:18.232345 21228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:45:18.248939 21239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:18.249172 21238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:18.249635 21242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:45:18.250123 21228 server_base.cc:1034] running on GCE node
I20260806 02:45:18.250962 21228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:45:18.252609 21228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:45:18.253897 21228 hybrid_clock.cc:648] HybridClock initialized: now 1785984318253777 us; error 35 us; skew 500 ppm
I20260806 02:45:18.254222 21228 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 02:45:18.258186 21228 webserver.cc:460] Webserver started at http://127.20.60.60:46389/ using document root <none> and password file <none>
I20260806 02:45:18.258984 21228 fs_manager.cc:359] Metadata directory not provided
I20260806 02:45:18.258908 21129 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } attempt: 3
I20260806 02:45:18.259183 21228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:45:18.259670 21228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
W20260806 02:45:18.261039 21129 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.60.60:42309: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:18.263500 21228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data/instance:
uuid: "894be089543b4aa29b3edfcc5f3085e9"
format_stamp: "Formatted at 2026-08-06 02:45:18 on dist-test-slave-bw88"
I20260806 02:45:18.264456 21228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/wal/instance:
uuid: "894be089543b4aa29b3edfcc5f3085e9"
format_stamp: "Formatted at 2026-08-06 02:45:18 on dist-test-slave-bw88"
I20260806 02:45:18.271126 21228 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.003s
I20260806 02:45:18.276117 21249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:18.277886 21228 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260806 02:45:18.278121 21228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data,/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/wal
uuid: "894be089543b4aa29b3edfcc5f3085e9"
format_stamp: "Formatted at 2026-08-06 02:45:18 on dist-test-slave-bw88"
I20260806 02:45:18.278402 21228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:45:18.301599 21228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:45:18.302938 21228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:45:18.303493 21228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:45:18.343629 21228 rpc_server.cc:307] RPC server started. Bound to: 127.20.60.60:42309
I20260806 02:45:18.343732 21315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.60.60:42309 every 8 connection(s)
I20260806 02:45:18.345691 21228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data/info.pb
I20260806 02:45:18.349146 20720 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu as pid 21228
I20260806 02:45:18.349478 20720 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/wal/instance
I20260806 02:45:18.350524 21316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:18.358876 21316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:45:18.379444 21316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:45:18.391098 21316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:45:18.412014 21316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Bootstrap starting.
I20260806 02:45:18.419410 21316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:18.421645 21316 log.cc:824] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Log is configured to *not* fsync() on all Append() calls
I20260806 02:45:18.427032 21316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: No bootstrap required, opened a new log
I20260806 02:45:18.438280 21129 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } attempt: 4
I20260806 02:45:18.441990 21316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:18.442834 21316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:18.443151 21316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 894be089543b4aa29b3edfcc5f3085e9, State: Initialized, Role: FOLLOWER
I20260806 02:45:18.447288 21316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:18.450546 21329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:18.451409 21329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:18.453760 21316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:45:18.454743 21129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Bootstrap starting.
I20260806 02:45:18.460312 21226 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } attempt: 5
I20260806 02:45:18.461254 21129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:18.463691 21129 log.cc:824] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Log is configured to *not* fsync() on all Append() calls
I20260806 02:45:18.467540 21129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: No bootstrap required, opened a new log
I20260806 02:45:18.482043 21226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Bootstrap starting.
I20260806 02:45:18.480772 21329 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:18.482822 21329 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:18.484440 21129 raft_consensus.cc:348] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:18.485006 21129 raft_consensus.cc:374] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:18.485379 21129 raft_consensus.cc:729] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a225e002d607451a94b3686f10aeea9a, State: Initialized, Role: FOLLOWER
I20260806 02:45:18.486541 21129 consensus_queue.cc:260] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:18.489275 21226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:18.491310 21329 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
I20260806 02:45:18.491789 21226 log.cc:824] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Log is configured to *not* fsync() on all Append() calls
I20260806 02:45:18.501468 21346 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:18.502189 21346 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:18.504629 21096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "894be089543b4aa29b3edfcc5f3085e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a225e002d607451a94b3686f10aeea9a" is_pre_election: true
I20260806 02:45:18.506098 21096 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 894be089543b4aa29b3edfcc5f3085e9 in term 0.
I20260806 02:45:18.507053 21129 sys_catalog.cc:564] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:45:18.507537 21253 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a; no voters:
I20260806 02:45:18.508320 21329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:18.508548 21329 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:18.508782 21329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 0 FOLLOWER]: Advancing to term 1
W20260806 02:45:18.514088 21071 tablet.cc:2369] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:45:18.513929 21329 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:18.516224 21096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "894be089543b4aa29b3edfcc5f3085e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a225e002d607451a94b3686f10aeea9a"
I20260806 02:45:18.516538 21096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:18.522524 21096 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 894be089543b4aa29b3edfcc5f3085e9 in term 1.
I20260806 02:45:18.524041 21329 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 1 election: Requested vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
I20260806 02:45:18.525360 21253 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a; no voters:
I20260806 02:45:18.526093 21329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:18.538481 21329 raft_consensus.cc:686] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 1 LEADER]: Becoming Leader. State: Replica: 894be089543b4aa29b3edfcc5f3085e9, State: Running, Role: LEADER
I20260806 02:45:18.539916 21226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: No bootstrap required, opened a new log
I20260806 02:45:18.541057 21329 consensus_queue.cc:237] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
W20260806 02:45:18.557200 21261 tablet.cc:2369] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:45:18.559104 21226 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:18.559669 21226 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:18.534441 21228 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: OK, instance_id { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" instance_seqno: 1785984317279177 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 02:45:18.559880 21226 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfdbf4ef60014ea98bbcf9b6a9033f22, State: Initialized, Role: FOLLOWER
I20260806 02:45:18.560245 21228 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:45:18.560633 21226 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:18.563081 21356 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:18.564252 21356 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:18.564819 21192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "894be089543b4aa29b3edfcc5f3085e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22"
I20260806 02:45:18.564954 21194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "894be089543b4aa29b3edfcc5f3085e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" is_pre_election: true
I20260806 02:45:18.565973 21194 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 894be089543b4aa29b3edfcc5f3085e9 in term 0.
I20260806 02:45:18.564812 21226 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 02:45:18.586404 21349 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:45:18.586970 21349 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:45:18.586723 21353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 894be089543b4aa29b3edfcc5f3085e9. Latest consensus state: current_term: 1 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:18.587921 21353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:18.599426 21376 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:45:18.604099 21376 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:45:18.670274 21192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:18.670471 21096 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 1 FOLLOWER]: Refusing update from remote peer 894be089543b4aa29b3edfcc5f3085e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:45:18.672255 21353 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:18.677814 21192 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 1 FOLLOWER]: Refusing update from remote peer 894be089543b4aa29b3edfcc5f3085e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:45:18.686653 21353 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:18.702492 21346 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 894be089543b4aa29b3edfcc5f3085e9. Latest consensus state: current_term: 1 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:18.703040 21346 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:18.714874 21329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:18.715509 21329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:18.727483 21376 catalog_manager.cc:1347] Generated new cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:18.727701 21376 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:45:18.732134 21353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:18.732455 21353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:18.736472 21356 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 894be089543b4aa29b3edfcc5f3085e9. Latest consensus state: current_term: 1 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:18.737126 21356 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:18.739609 21394 mvcc.cc:205] Tried to move back new op lower bound from 7315391769179324416 to 7315391768763592704. Current Snapshot: MvccSnapshot[applied={T|T < 7315391769179324416}]
I20260806 02:45:18.753063 21356 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:18.753366 21356 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:18.784785 21346 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:18.785159 21346 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:18.720721 21045 master_runner.cc:186] Error getting master registration for 127.20.60.61:36797: OK, instance_id { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" instance_seqno: 1785984317748482 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 02:45:18.807036 21045 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:45:18.815383 21395 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:45:18.828768 21395 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:45:18.847807 21376 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:45:18.850982 21376 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:45:18.860539 21410 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:18.860715 21410 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: loading cluster ID for follower catalog manager: success
I20260806 02:45:18.865839 21410 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: acquiring CA information for follower catalog manager: success
W20260806 02:45:18.875981 21410 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260806 02:45:18.876250 21376 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Generated new TSK 0
I20260806 02:45:18.877650 21376 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:45:18.880069 21376 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260806 02:45:18.881914 21376 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260806 02:45:18.906847 20720 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
/tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.60.1:0
--local_ip_for_outbound_sockets=127.20.60.1
--webserver_interface=127.20.60.1
--webserver_port=0
--tserver_master_addrs=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--builtin_ntp_servers=127.20.60.20:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20260806 02:45:19.736129 21415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:45:19.736629 21415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:45:19.736809 21415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:45:19.752218 21415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:45:19.752468 21415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.60.1
I20260806 02:45:19.762430 21419 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 894be089543b4aa29b3edfcc5f3085e9)
I20260806 02:45:19.763037 21419 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:19.765573 21419 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
I20260806 02:45:19.766215 21192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a225e002d607451a94b3686f10aeea9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" is_pre_election: true
I20260806 02:45:19.766644 21285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a225e002d607451a94b3686f10aeea9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "894be089543b4aa29b3edfcc5f3085e9" is_pre_election: true
I20260806 02:45:19.767714 21068 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a225e002d607451a94b3686f10aeea9a; no voters: 894be089543b4aa29b3edfcc5f3085e9, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:19.768529 21419 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:45:19.772344 21415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.60.20:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.60.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.60.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.60.1
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 01:55:54 UTC on ccycloud.quasar-csdhqu.root.comops.site
build id 4569536
ASAN enabled
I20260806 02:45:19.774389 21415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:45:19.777153 21415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:45:19.796571 21422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:19.796813 21425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:19.796886 21421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:45:19.798223 21415 server_base.cc:1034] running on GCE node
I20260806 02:45:19.799026 21415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:45:19.800129 21415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:45:19.801496 21415 hybrid_clock.cc:648] HybridClock initialized: now 1785984319801287 us; error 50 us; skew 500 ppm
I20260806 02:45:19.801894 21415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 02:45:19.804982 21415 webserver.cc:460] Webserver started at http://127.20.60.1:45553/ using document root <none> and password file <none>
I20260806 02:45:19.805912 21415 fs_manager.cc:359] Metadata directory not provided
I20260806 02:45:19.806034 21415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:45:19.806401 21415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:45:19.809528 21415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/data/instance:
uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
format_stamp: "Formatted at 2026-08-06 02:45:19 on dist-test-slave-bw88"
I20260806 02:45:19.810338 21415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/instance:
uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
format_stamp: "Formatted at 2026-08-06 02:45:19 on dist-test-slave-bw88"
I20260806 02:45:19.816715 21415 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260806 02:45:19.821475 21433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:19.822958 21415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 02:45:19.823163 21415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal
uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
format_stamp: "Formatted at 2026-08-06 02:45:19 on dist-test-slave-bw88"
I20260806 02:45:19.823621 21415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:45:19.831985 21395 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:19.832298 21395 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: loading cluster ID for follower catalog manager: success
I20260806 02:45:19.836826 21395 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: acquiring CA information for follower catalog manager: success
I20260806 02:45:19.839805 21395 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 02:45:19.858443 21415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:45:19.859783 21415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:45:19.860332 21415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:45:19.865775 21415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:45:19.869506 21415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:45:19.869596 21415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:19.869820 21415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:45:19.869990 21415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:19.880559 21410 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 02:45:19.945101 21415 rpc_server.cc:307] RPC server started. Bound to: 127.20.60.1:36593
I20260806 02:45:19.945276 21580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.60.1:36593 every 8 connection(s)
I20260806 02:45:19.946717 21415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/data/info.pb
I20260806 02:45:19.949849 20720 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu as pid 21415
I20260806 02:45:19.950145 20720 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/instance
I20260806 02:45:19.960829 20720 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
/tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.60.2:0
--local_ip_for_outbound_sockets=127.20.60.2
--webserver_interface=127.20.60.2
--webserver_port=0
--tserver_master_addrs=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--builtin_ntp_servers=127.20.60.20:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260806 02:45:20.015409 21581 heartbeater.cc:347] Connected to a master server at 127.20.60.60:42309
I20260806 02:45:20.015973 21581 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:20.017644 21581 heartbeater.cc:511] Master 127.20.60.60:42309 requested a full tablet report, sending...
I20260806 02:45:20.018782 21582 heartbeater.cc:347] Connected to a master server at 127.20.60.62:33227
I20260806 02:45:20.018989 21582 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:20.019799 21583 heartbeater.cc:347] Connected to a master server at 127.20.60.61:36797
I20260806 02:45:20.019960 21582 heartbeater.cc:511] Master 127.20.60.62:33227 requested a full tablet report, sending...
I20260806 02:45:20.020057 21583 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:20.022181 21583 heartbeater.cc:511] Master 127.20.60.61:36797 requested a full tablet report, sending...
I20260806 02:45:20.024451 21178 ts_manager.cc:194] Registered new tserver with Master: 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:20.023942 21084 ts_manager.cc:194] Registered new tserver with Master: 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:20.024200 21274 ts_manager.cc:194] Registered new tserver with Master: 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:20.027774 21274 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.60.1:45273
I20260806 02:45:20.340121 21593 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 894be089543b4aa29b3edfcc5f3085e9)
I20260806 02:45:20.340487 21593 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:20.342914 21593 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
I20260806 02:45:20.342994 21096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a225e002d607451a94b3686f10aeea9a" is_pre_election: true
I20260806 02:45:20.343732 21285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "894be089543b4aa29b3edfcc5f3085e9" is_pre_election: true
I20260806 02:45:20.344394 21163 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a
I20260806 02:45:20.345093 21593 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:45:20.380339 21594 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 894be089543b4aa29b3edfcc5f3085e9)
I20260806 02:45:20.380551 21594 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:20.381573 21594 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
I20260806 02:45:20.382329 21192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a225e002d607451a94b3686f10aeea9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" is_pre_election: true
I20260806 02:45:20.382499 21285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a225e002d607451a94b3686f10aeea9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "894be089543b4aa29b3edfcc5f3085e9" is_pre_election: true
I20260806 02:45:20.382683 21192 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a225e002d607451a94b3686f10aeea9a in term 1.
I20260806 02:45:20.383318 21064 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22; no voters:
I20260806 02:45:20.383637 21594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 02:45:20.383778 21594 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 894be089543b4aa29b3edfcc5f3085e9)
I20260806 02:45:20.383906 21594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 1 FOLLOWER]: Advancing to term 2
I20260806 02:45:20.451789 21594 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:20.453321 21594 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 2 election: Requested vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
I20260806 02:45:20.454303 21285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a225e002d607451a94b3686f10aeea9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "894be089543b4aa29b3edfcc5f3085e9"
I20260806 02:45:20.454576 21192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a225e002d607451a94b3686f10aeea9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22"
I20260806 02:45:20.455499 21064 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a225e002d607451a94b3686f10aeea9a; no voters: 894be089543b4aa29b3edfcc5f3085e9, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:20.456084 21594 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 02:45:20.495229 21096 raft_consensus.cc:1226] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 2 FOLLOWER]: Rejecting Update request from peer 894be089543b4aa29b3edfcc5f3085e9 for earlier term 1. Current term is 2. Ops: []
I20260806 02:45:20.496268 21596 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 02:45:20.497668 21597 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 1 LEADER]: Stepping down as leader of term 1
I20260806 02:45:20.497885 21597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 894be089543b4aa29b3edfcc5f3085e9, State: Running, Role: LEADER
I20260806 02:45:20.498512 21597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:20.499351 21597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 1 FOLLOWER]: Advancing to term 2
I20260806 02:45:20.532039 21581 heartbeater.cc:503] Master 127.20.60.60:42309 was elected leader, sending a full tablet report...
I20260806 02:45:20.590597 21349 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 02:45:20.608817 21590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:45:20.609237 21590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:45:20.609444 21590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:45:20.619019 21590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:45:20.619341 21590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.60.2
I20260806 02:45:20.631906 21590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.60.20:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.60.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.60.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.60.2
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 01:55:54 UTC on ccycloud.quasar-csdhqu.root.comops.site
build id 4569536
ASAN enabled
I20260806 02:45:20.633569 21590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:45:20.636004 21590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:45:20.657444 21602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:20.657440 21607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:20.657318 21603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:45:20.661643 21590 server_base.cc:1034] running on GCE node
I20260806 02:45:20.662413 21590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:45:20.663523 21590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:45:20.664834 21590 hybrid_clock.cc:648] HybridClock initialized: now 1785984320664681 us; error 63 us; skew 500 ppm
I20260806 02:45:20.665264 21590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 02:45:20.669200 21590 webserver.cc:460] Webserver started at http://127.20.60.2:44429/ using document root <none> and password file <none>
I20260806 02:45:20.670135 21590 fs_manager.cc:359] Metadata directory not provided
I20260806 02:45:20.670295 21590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:45:20.670692 21590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:45:20.674417 21590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/data/instance:
uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
format_stamp: "Formatted at 2026-08-06 02:45:20 on dist-test-slave-bw88"
I20260806 02:45:20.675395 21590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/instance:
uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
format_stamp: "Formatted at 2026-08-06 02:45:20 on dist-test-slave-bw88"
I20260806 02:45:20.682667 21590 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 02:45:20.687289 21616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:20.689253 21590 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260806 02:45:20.689559 21590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal
uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
format_stamp: "Formatted at 2026-08-06 02:45:20 on dist-test-slave-bw88"
I20260806 02:45:20.689818 21590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:45:20.748620 21590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:45:20.749951 21590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:45:20.750442 21590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:45:20.752162 21590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:45:20.755458 21590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:45:20.755676 21590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:20.755929 21590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:45:20.756189 21590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:20.835204 21590 rpc_server.cc:307] RPC server started. Bound to: 127.20.60.2:40655
I20260806 02:45:20.835356 21760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.60.2:40655 every 8 connection(s)
I20260806 02:45:20.836968 21590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/data/info.pb
I20260806 02:45:20.842555 20720 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu as pid 21590
I20260806 02:45:20.842797 20720 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/instance
I20260806 02:45:20.891626 21762 heartbeater.cc:347] Connected to a master server at 127.20.60.62:33227
I20260806 02:45:20.892046 21761 heartbeater.cc:347] Connected to a master server at 127.20.60.60:42309
I20260806 02:45:20.892118 21762 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:20.892227 21761 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:20.893910 21761 heartbeater.cc:511] Master 127.20.60.60:42309 requested a full tablet report, sending...
I20260806 02:45:20.893924 21762 heartbeater.cc:511] Master 127.20.60.62:33227 requested a full tablet report, sending...
I20260806 02:45:20.895836 21274 ts_manager.cc:194] Registered new tserver with Master: 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:20.895843 21084 ts_manager.cc:194] Registered new tserver with Master: 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:20.898106 21763 heartbeater.cc:347] Connected to a master server at 127.20.60.61:36797
I20260806 02:45:20.898327 21763 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:20.899230 21763 heartbeater.cc:511] Master 127.20.60.61:36797 requested a full tablet report, sending...
I20260806 02:45:20.900676 21178 ts_manager.cc:194] Registered new tserver with Master: 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:20.902014 21774 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 894be089543b4aa29b3edfcc5f3085e9)
I20260806 02:45:20.902357 21774 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:20.903146 21774 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
I20260806 02:45:20.903826 21096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a225e002d607451a94b3686f10aeea9a" is_pre_election: true
I20260806 02:45:20.903823 21285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "894be089543b4aa29b3edfcc5f3085e9" is_pre_election: true
I20260806 02:45:20.904145 21096 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bfdbf4ef60014ea98bbcf9b6a9033f22 in current term 2: Already voted for candidate a225e002d607451a94b3686f10aeea9a in this term.
I20260806 02:45:20.904121 21285 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfdbf4ef60014ea98bbcf9b6a9033f22 in term 2.
I20260806 02:45:20.904641 21163 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9, bfdbf4ef60014ea98bbcf9b6a9033f22; no voters:
I20260806 02:45:20.905019 21774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 02:45:20.905175 21774 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 894be089543b4aa29b3edfcc5f3085e9)
I20260806 02:45:20.905295 21774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 1 FOLLOWER]: Advancing to term 2
I20260806 02:45:20.907706 21774 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:20.908522 21774 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 2 election: Requested vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
I20260806 02:45:20.909036 21285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "894be089543b4aa29b3edfcc5f3085e9"
I20260806 02:45:20.909145 21096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a225e002d607451a94b3686f10aeea9a"
I20260806 02:45:20.909534 21096 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bfdbf4ef60014ea98bbcf9b6a9033f22 in current term 2: Already voted for candidate a225e002d607451a94b3686f10aeea9a in this term.
I20260806 02:45:20.913134 21285 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfdbf4ef60014ea98bbcf9b6a9033f22 in term 2.
I20260806 02:45:20.913633 21163 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9, bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: a225e002d607451a94b3686f10aeea9a
I20260806 02:45:20.914013 21774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 02:45:20.914872 21774 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 2 LEADER]: Becoming Leader. State: Replica: bfdbf4ef60014ea98bbcf9b6a9033f22, State: Running, Role: LEADER
I20260806 02:45:20.915469 21774 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:20.917475 21775 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfdbf4ef60014ea98bbcf9b6a9033f22. Latest consensus state: current_term: 2 leader_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:20.918227 21775 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:20.919976 21777 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:45:20.923209 21777 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:45:20.924685 21777 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:20.924841 21777 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:45:20.926661 21777 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:45:20.927870 21777 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Loaded TSK: 0
I20260806 02:45:20.928910 21777 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:45:20.929881 21777 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260806 02:45:20.931294 21777 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260806 02:45:20.942744 20720 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
/tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.60.3:0
--local_ip_for_outbound_sockets=127.20.60.3
--webserver_interface=127.20.60.3
--webserver_port=0
--tserver_master_addrs=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--builtin_ntp_servers=127.20.60.20:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260806 02:45:21.159510 21779 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:21.159780 21779 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:21.160717 21779 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
I20260806 02:45:21.161415 21285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a225e002d607451a94b3686f10aeea9a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "894be089543b4aa29b3edfcc5f3085e9" is_pre_election: true
I20260806 02:45:21.161500 21192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a225e002d607451a94b3686f10aeea9a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" is_pre_election: true
I20260806 02:45:21.161782 21285 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a225e002d607451a94b3686f10aeea9a in term 2.
I20260806 02:45:21.162303 21068 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a; no voters: bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:21.162743 21779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 02:45:21.162978 21779 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:21.163134 21779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 2 FOLLOWER]: Advancing to term 3
I20260806 02:45:21.169265 21779 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:21.170568 21779 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 3 election: Requested vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
I20260806 02:45:21.171010 21192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a225e002d607451a94b3686f10aeea9a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22"
I20260806 02:45:21.171217 21285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a225e002d607451a94b3686f10aeea9a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "894be089543b4aa29b3edfcc5f3085e9"
I20260806 02:45:21.171519 21285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 2 FOLLOWER]: Advancing to term 3
I20260806 02:45:21.175463 21285 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a225e002d607451a94b3686f10aeea9a in term 3.
I20260806 02:45:21.176153 21068 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a; no voters: bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:21.176576 21779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 3 FOLLOWER]: Leader election won for term 3
I20260806 02:45:21.177507 21779 raft_consensus.cc:686] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 3 LEADER]: Becoming Leader. State: Replica: a225e002d607451a94b3686f10aeea9a, State: Running, Role: LEADER
I20260806 02:45:21.178035 21779 consensus_queue.cc:237] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:21.180748 21780 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a225e002d607451a94b3686f10aeea9a. Latest consensus state: current_term: 3 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:21.181547 21780 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:21.182723 21782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:45:21.185721 21782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:45:21.187098 21782 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:21.187203 21782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:45:21.189164 21782 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:45:21.190783 21782 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Loaded TSK: 0
I20260806 02:45:21.192260 21782 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:45:21.193395 21782 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260806 02:45:21.194638 21782 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260806 02:45:21.346700 21285 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 3 FOLLOWER]: Rejecting Update request from peer bfdbf4ef60014ea98bbcf9b6a9033f22 for earlier term 2. Current term is 3. Ops: []
I20260806 02:45:21.347888 21775 consensus_queue.cc:1046] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 02:45:21.348497 21775 raft_consensus.cc:3032] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 2 LEADER]: Stepping down as leader of term 2
I20260806 02:45:21.348726 21775 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bfdbf4ef60014ea98bbcf9b6a9033f22, State: Running, Role: LEADER
I20260806 02:45:21.349273 21775 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:21.350188 21775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 2 FOLLOWER]: Advancing to term 3
I20260806 02:45:21.401461 21084 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.60.2:36491
W20260806 02:45:21.503070 21778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:45:21.503456 21778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:45:21.503628 21778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:45:21.512827 21778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:45:21.513131 21778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.60.3
I20260806 02:45:21.524675 21778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.60.20:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.60.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.60.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.60.3
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 01:55:54 UTC on ccycloud.quasar-csdhqu.root.comops.site
build id 4569536
ASAN enabled
I20260806 02:45:21.526239 21778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:45:21.528290 21778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 02:45:21.533615 21583 heartbeater.cc:503] Master 127.20.60.61:36797 was elected leader, sending a full tablet report...
W20260806 02:45:21.547186 21788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:21.548277 21791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:21.548273 21794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:45:21.548873 21778 server_base.cc:1034] running on GCE node
I20260806 02:45:21.549605 21778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:45:21.550612 21778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:45:21.551957 21778 hybrid_clock.cc:648] HybridClock initialized: now 1785984321551777 us; error 41 us; skew 500 ppm
I20260806 02:45:21.552342 21778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 02:45:21.555699 21778 webserver.cc:460] Webserver started at http://127.20.60.3:35641/ using document root <none> and password file <none>
I20260806 02:45:21.556635 21778 fs_manager.cc:359] Metadata directory not provided
I20260806 02:45:21.556825 21778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:45:21.557200 21778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:45:21.560087 21778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/data/instance:
uuid: "8ebd96b903b944f3b06851324b737bd5"
format_stamp: "Formatted at 2026-08-06 02:45:21 on dist-test-slave-bw88"
I20260806 02:45:21.560871 21778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/instance:
uuid: "8ebd96b903b944f3b06851324b737bd5"
format_stamp: "Formatted at 2026-08-06 02:45:21 on dist-test-slave-bw88"
I20260806 02:45:21.567584 21778 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 02:45:21.572065 21802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:21.573685 21778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:45:21.573943 21778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal
uuid: "8ebd96b903b944f3b06851324b737bd5"
format_stamp: "Formatted at 2026-08-06 02:45:21 on dist-test-slave-bw88"
I20260806 02:45:21.574290 21778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:45:21.613759 21778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:45:21.615038 21778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:45:21.615561 21778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:45:21.616972 21778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:45:21.619378 21778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:45:21.619463 21778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:21.619700 21778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:45:21.619840 21778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:21.685065 21778 rpc_server.cc:307] RPC server started. Bound to: 127.20.60.3:43883
I20260806 02:45:21.685220 21949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.60.3:43883 every 8 connection(s)
I20260806 02:45:21.686682 21778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/data/info.pb
I20260806 02:45:21.689059 20720 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu as pid 21778
I20260806 02:45:21.689261 20720 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/instance
I20260806 02:45:21.728560 21192 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 3 FOLLOWER]: Refusing update from remote peer a225e002d607451a94b3686f10aeea9a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260806 02:45:21.728905 21192 pending_rounds.cc:76] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Aborting all ops after (but not including) 4
I20260806 02:45:21.729092 21192 pending_rounds.cc:97] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260806 02:45:21.729372 21192 raft_consensus.cc:2866] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260806 02:45:21.730472 21962 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 02:45:21.733268 21775 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a225e002d607451a94b3686f10aeea9a. Latest consensus state: current_term: 3 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:21.733664 21775 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:21.741541 21285 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 3 FOLLOWER]: Refusing update from remote peer a225e002d607451a94b3686f10aeea9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260806 02:45:21.741381 21963 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:21.741814 21963 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:21.744889 21951 heartbeater.cc:347] Connected to a master server at 127.20.60.62:33227
I20260806 02:45:21.745396 21951 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:21.746999 21951 heartbeater.cc:511] Master 127.20.60.62:33227 requested a full tablet report, sending...
I20260806 02:45:21.747648 21775 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:21.748020 21775 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:21.749704 21952 heartbeater.cc:347] Connected to a master server at 127.20.60.61:36797
I20260806 02:45:21.749909 21952 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:21.750336 21963 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260806 02:45:21.750897 21952 heartbeater.cc:511] Master 127.20.60.61:36797 requested a full tablet report, sending...
I20260806 02:45:21.751366 21950 heartbeater.cc:347] Connected to a master server at 127.20.60.60:42309
I20260806 02:45:21.751581 21950 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:21.752255 21950 heartbeater.cc:511] Master 127.20.60.60:42309 requested a full tablet report, sending...
I20260806 02:45:21.753300 21177 ts_manager.cc:194] Registered new tserver with Master: 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:21.755373 21084 ts_manager.cc:194] Registered new tserver with Master: 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:21.756915 21084 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.60.3:34437
I20260806 02:45:21.758744 21972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a225e002d607451a94b3686f10aeea9a. Latest consensus state: current_term: 3 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:21.759114 21274 ts_manager.cc:194] Registered new tserver with Master: 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:21.759243 21972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:21.759503 21972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:21.759815 21972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:21.767551 20720 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:45:21.831756 21084 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_910edc3d61454438a69af3c32d9b721f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:21.830637 21082 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_8b2d472c87014c6086b51a9946708d3a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:21.830731 21083 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_358a74d95709455e931a3b3c82ab656e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:21.832697 21081 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_54a040201f184199874cbda298addd03"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:21.834142 21080 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_00f2c397972f4d8e828d61d7821abc00"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:21.835632 21077 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_c94f1c901de74f71bc21408cd63fa3ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:21.839847 21073 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_0e9bb7fa1ed44ca0af8546b2c255fa62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:21.840790 21074 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_f36563aa9b94423ebebd9741529ba14c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:21.836381 21079 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_c2d5cb4cc8a44eaca15ba851ecc366c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:21.836689 21076 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_a7715d8e001f48b2899c3c2a56a18a06"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:45:21.850683 21082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8b2d472c87014c6086b51a9946708d3a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:45:21.867115 20720 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu pid 21132
I20260806 02:45:21.868667 21218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 02:45:21.904742 21762 heartbeater.cc:503] Master 127.20.60.62:33227 was elected leader, sending a full tablet report...
I20260806 02:45:22.033527 21582 heartbeater.cc:503] Master 127.20.60.62:33227 was elected leader, sending a full tablet report...
I20260806 02:45:22.053043 20720 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu with pid 21132
W20260806 02:45:22.097213 21763 heartbeater.cc:644] Failed to heartbeat to 127.20.60.61:36797 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.20.60.61:36797 (error 108)
W20260806 02:45:22.098884 21583 heartbeater.cc:644] Failed to heartbeat to 127.20.60.61:36797 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.20.60.61:36797 (error 108)
I20260806 02:45:22.217089 21971 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:22.218125 21971 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: LEADER
W20260806 02:45:22.256791 21952 heartbeater.cc:644] Failed to heartbeat to 127.20.60.61:36797 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:45:22.260629 21951 heartbeater.cc:503] Master 127.20.60.62:33227 was elected leader, sending a full tablet report...
I20260806 02:45:22.310041 21082 catalog_manager.cc:2148] created HMS catalog entry for table [id=b8a5ee6530364be1ae88e64a44dcc022]
W20260806 02:45:22.312095 21083 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_358a74d95709455e931a3b3c82ab656e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 02:45:22.312184 21084 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_910edc3d61454438a69af3c32d9b721f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 02:45:22.312435 21080 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_00f2c397972f4d8e828d61d7821abc00 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 02:45:22.312186 21081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_54a040201f184199874cbda298addd03 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 02:45:22.313436 21077 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_c94f1c901de74f71bc21408cd63fa3ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 02:45:22.313727 21076 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_a7715d8e001f48b2899c3c2a56a18a06 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 02:45:22.314709 21079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_c2d5cb4cc8a44eaca15ba851ecc366c8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 02:45:22.315659 21074 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f36563aa9b94423ebebd9741529ba14c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 02:45:22.316540 21073 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 02:45:22.336942 21064 consensus_peers.cc:487] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a -> Peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Couldn't send request to peer bfdbf4ef60014ea98bbcf9b6a9033f22. Status: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 02:45:22.341864 21082 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58094:
table { table_name: "default.Table_8b2d472c87014c6086b51a9946708d3a" } new_table_name: "default.Table_e24bd493305d46a4977ed544ce47cc83" modify_external_catalogs: true
I20260806 02:45:22.373780 21083 catalog_manager.cc:2148] created HMS catalog entry for table [id=370d5895c35f4c96878270e71cac2443]
I20260806 02:45:22.386861 21865 tablet_service.cc:1467] Processing CreateTablet for tablet a12d9a1b4ebc496a86720ee06efd9c4e (DEFAULT_TABLE table=default.table_8b2d472c87014c6086b51a9946708d3a [id=b8a5ee6530364be1ae88e64a44dcc022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.388746 21865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a12d9a1b4ebc496a86720ee06efd9c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.393751 21683 tablet_service.cc:1467] Processing CreateTablet for tablet a12d9a1b4ebc496a86720ee06efd9c4e (DEFAULT_TABLE table=default.table_8b2d472c87014c6086b51a9946708d3a [id=b8a5ee6530364be1ae88e64a44dcc022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.396689 21683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a12d9a1b4ebc496a86720ee06efd9c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.398573 21498 tablet_service.cc:1467] Processing CreateTablet for tablet a12d9a1b4ebc496a86720ee06efd9c4e (DEFAULT_TABLE table=default.table_8b2d472c87014c6086b51a9946708d3a [id=b8a5ee6530364be1ae88e64a44dcc022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.400033 21498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a12d9a1b4ebc496a86720ee06efd9c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.407511 21083 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58094:
table { table_name: "default.Table_358a74d95709455e931a3b3c82ab656e" } new_table_name: "default.Table_8797f6f61c444bd0a4e4555f6ca2fe30" modify_external_catalogs: true
I20260806 02:45:22.418239 21863 tablet_service.cc:1467] Processing CreateTablet for tablet c12fa3c68f52497a9ed0fe9398a46b98 (DEFAULT_TABLE table=default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.418592 21495 tablet_service.cc:1467] Processing CreateTablet for tablet c12fa3c68f52497a9ed0fe9398a46b98 (DEFAULT_TABLE table=default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.418826 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c12fa3c68f52497a9ed0fe9398a46b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.419189 21495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c12fa3c68f52497a9ed0fe9398a46b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.421917 21682 tablet_service.cc:1467] Processing CreateTablet for tablet c12fa3c68f52497a9ed0fe9398a46b98 (DEFAULT_TABLE table=default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.422564 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c12fa3c68f52497a9ed0fe9398a46b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.432075 22046 tablet_bootstrap.cc:492] T c12fa3c68f52497a9ed0fe9398a46b98 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:22.432487 22045 tablet_bootstrap.cc:492] T a12d9a1b4ebc496a86720ee06efd9c4e P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:22.432595 22047 tablet_bootstrap.cc:492] T c12fa3c68f52497a9ed0fe9398a46b98 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:22.436129 22046 tablet_bootstrap.cc:654] T c12fa3c68f52497a9ed0fe9398a46b98 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.436533 22045 tablet_bootstrap.cc:654] T a12d9a1b4ebc496a86720ee06efd9c4e P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.436681 22047 tablet_bootstrap.cc:654] T c12fa3c68f52497a9ed0fe9398a46b98 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.437937 22046 log.cc:824] T c12fa3c68f52497a9ed0fe9398a46b98 P 8ebd96b903b944f3b06851324b737bd5: Log is configured to *not* fsync() on all Append() calls
I20260806 02:45:22.437943 22047 log.cc:824] T c12fa3c68f52497a9ed0fe9398a46b98 P 88dc3ab284a64961b1fbdb5c53d1267f: Log is configured to *not* fsync() on all Append() calls
I20260806 02:45:22.438287 22045 log.cc:824] T a12d9a1b4ebc496a86720ee06efd9c4e P 48b0c1e2106b4d9b941929ddc6d97dd5: Log is configured to *not* fsync() on all Append() calls
I20260806 02:45:22.445190 22046 tablet_bootstrap.cc:492] T c12fa3c68f52497a9ed0fe9398a46b98 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:22.445570 22046 ts_tablet_manager.cc:1397] T c12fa3c68f52497a9ed0fe9398a46b98 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260806 02:45:22.446972 22047 tablet_bootstrap.cc:492] T c12fa3c68f52497a9ed0fe9398a46b98 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:22.447268 22047 ts_tablet_manager.cc:1397] T c12fa3c68f52497a9ed0fe9398a46b98 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20260806 02:45:22.451964 21084 catalog_manager.cc:2148] created HMS catalog entry for table [id=1bbd1184e7754c5bb9ffdf82791c73fb]
I20260806 02:45:22.457784 22047 raft_consensus.cc:348] T c12fa3c68f52497a9ed0fe9398a46b98 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.458297 22047 raft_consensus.cc:374] T c12fa3c68f52497a9ed0fe9398a46b98 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.458424 22047 raft_consensus.cc:729] T c12fa3c68f52497a9ed0fe9398a46b98 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.459221 22047 consensus_queue.cc:260] T c12fa3c68f52497a9ed0fe9398a46b98 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.462291 21084 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58094:
table { table_name: "default.Table_910edc3d61454438a69af3c32d9b721f" } new_table_name: "default.Table_e41f56f2e4a1439ca6b8f2bca460a670" modify_external_catalogs: true
I20260806 02:45:22.471565 22047 ts_tablet_manager.cc:1428] T c12fa3c68f52497a9ed0fe9398a46b98 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.024s user 0.012s sys 0.012s
I20260806 02:45:22.472059 22047 tablet_bootstrap.cc:492] T a12d9a1b4ebc496a86720ee06efd9c4e P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:22.473941 22047 tablet_bootstrap.cc:654] T a12d9a1b4ebc496a86720ee06efd9c4e P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.482730 22047 tablet_bootstrap.cc:492] T a12d9a1b4ebc496a86720ee06efd9c4e P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:22.482951 22047 ts_tablet_manager.cc:1397] T a12d9a1b4ebc496a86720ee06efd9c4e P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260806 02:45:22.484393 22047 raft_consensus.cc:348] T a12d9a1b4ebc496a86720ee06efd9c4e P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.484640 22047 raft_consensus.cc:374] T a12d9a1b4ebc496a86720ee06efd9c4e P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.484752 22047 raft_consensus.cc:729] T a12d9a1b4ebc496a86720ee06efd9c4e P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.485769 21498 tablet_service.cc:1467] Processing CreateTablet for tablet d0956f9352c84a069760fa70da9e8881 (DEFAULT_TABLE table=default.table_910edc3d61454438a69af3c32d9b721f [id=1bbd1184e7754c5bb9ffdf82791c73fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.486294 21498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0956f9352c84a069760fa70da9e8881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.491451 22045 tablet_bootstrap.cc:492] T a12d9a1b4ebc496a86720ee06efd9c4e P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:22.491815 22045 ts_tablet_manager.cc:1397] T a12d9a1b4ebc496a86720ee06efd9c4e P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.060s user 0.006s sys 0.003s
I20260806 02:45:22.493675 21863 tablet_service.cc:1467] Processing CreateTablet for tablet d0956f9352c84a069760fa70da9e8881 (DEFAULT_TABLE table=default.table_910edc3d61454438a69af3c32d9b721f [id=1bbd1184e7754c5bb9ffdf82791c73fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.494302 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0956f9352c84a069760fa70da9e8881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.496129 21080 catalog_manager.cc:2148] created HMS catalog entry for table [id=610932bdd3f94b4e886d586c28ff6249]
I20260806 02:45:22.485041 22047 consensus_queue.cc:260] T a12d9a1b4ebc496a86720ee06efd9c4e P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.502964 22047 ts_tablet_manager.cc:1428] T a12d9a1b4ebc496a86720ee06efd9c4e P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.020s user 0.002s sys 0.001s
I20260806 02:45:22.511580 21080 master_service.cc:946] ReplaceTablet: received request to replace tablet a12d9a1b4ebc496a86720ee06efd9c4e from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:22.514403 21498 tablet_service.cc:1467] Processing CreateTablet for tablet 043839a5eca143d5882f40faaab39692 (DEFAULT_TABLE table=default.table_00f2c397972f4d8e828d61d7821abc00 [id=610932bdd3f94b4e886d586c28ff6249]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.514978 21498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 043839a5eca143d5882f40faaab39692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.517982 21863 tablet_service.cc:1467] Processing CreateTablet for tablet 043839a5eca143d5882f40faaab39692 (DEFAULT_TABLE table=default.table_00f2c397972f4d8e828d61d7821abc00 [id=610932bdd3f94b4e886d586c28ff6249]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.518492 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 043839a5eca143d5882f40faaab39692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.520639 21080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Sending DeleteTablet for 3 replicas of tablet a12d9a1b4ebc496a86720ee06efd9c4e
I20260806 02:45:22.521914 21682 tablet_service.cc:1467] Processing CreateTablet for tablet d0956f9352c84a069760fa70da9e8881 (DEFAULT_TABLE table=default.table_910edc3d61454438a69af3c32d9b721f [id=1bbd1184e7754c5bb9ffdf82791c73fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.522007 21865 tablet_service.cc:1514] Processing DeleteTablet for tablet a12d9a1b4ebc496a86720ee06efd9c4e with delete_type TABLET_DATA_DELETED (replaced by tablet 80d6ff40e16d4114a2dc9658f6e63a3b) from {username='slave'} at 127.0.0.1:56372
I20260806 02:45:22.522379 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0956f9352c84a069760fa70da9e8881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.523123 21495 tablet_service.cc:1514] Processing DeleteTablet for tablet a12d9a1b4ebc496a86720ee06efd9c4e with delete_type TABLET_DATA_DELETED (replaced by tablet 80d6ff40e16d4114a2dc9658f6e63a3b) from {username='slave'} at 127.0.0.1:33104
I20260806 02:45:22.531551 21682 tablet_service.cc:1467] Processing CreateTablet for tablet 043839a5eca143d5882f40faaab39692 (DEFAULT_TABLE table=default.table_00f2c397972f4d8e828d61d7821abc00 [id=610932bdd3f94b4e886d586c28ff6249]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.532044 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 043839a5eca143d5882f40faaab39692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.544341 21498 tablet_service.cc:1467] Processing CreateTablet for tablet 80d6ff40e16d4114a2dc9658f6e63a3b (DEFAULT_TABLE table=default.table_8b2d472c87014c6086b51a9946708d3a [id=b8a5ee6530364be1ae88e64a44dcc022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.544893 21498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80d6ff40e16d4114a2dc9658f6e63a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.545591 21080 catalog_manager.cc:6565] ReplaceTablet: tablet a12d9a1b4ebc496a86720ee06efd9c4e deleted and replaced by tablet 80d6ff40e16d4114a2dc9658f6e63a3b
I20260806 02:45:22.549671 21080 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58094:
table { table_name: "default.Table_00f2c397972f4d8e828d61d7821abc00" } new_table_name: "default.Table_21221d53bf9f4c46a5ffe671a1ffe424" modify_external_catalogs: true
I20260806 02:45:22.551196 22047 tablet_bootstrap.cc:492] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:22.553071 22047 tablet_bootstrap.cc:654] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
W20260806 02:45:22.554056 21064 catalog_manager.cc:4821] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): delete failed for tablet a12d9a1b4ebc496a86720ee06efd9c4e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a12d9a1b4ebc496a86720ee06efd9c4e already in progress: creating tablet
I20260806 02:45:22.555089 21683 tablet_service.cc:1514] Processing DeleteTablet for tablet a12d9a1b4ebc496a86720ee06efd9c4e with delete_type TABLET_DATA_DELETED (replaced by tablet 80d6ff40e16d4114a2dc9658f6e63a3b) from {username='slave'} at 127.0.0.1:38404
I20260806 02:45:22.555446 21681 tablet_service.cc:1467] Processing CreateTablet for tablet 80d6ff40e16d4114a2dc9658f6e63a3b (DEFAULT_TABLE table=default.table_8b2d472c87014c6086b51a9946708d3a [id=b8a5ee6530364be1ae88e64a44dcc022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.556097 21681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80d6ff40e16d4114a2dc9658f6e63a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.557528 21081 catalog_manager.cc:2148] created HMS catalog entry for table [id=4c9f8d8c878a4b128b71e594b536267e]
I20260806 02:45:22.560508 22047 tablet_bootstrap.cc:492] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:22.560698 22047 ts_tablet_manager.cc:1397] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.005s
I20260806 02:45:22.561655 22047 raft_consensus.cc:348] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.561949 22047 raft_consensus.cc:374] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.562043 22047 raft_consensus.cc:729] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.565266 21081 master_service.cc:946] ReplaceTablet: received request to replace tablet c12fa3c68f52497a9ed0fe9398a46b98 from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:22.566461 21863 tablet_service.cc:1467] Processing CreateTablet for tablet 80d6ff40e16d4114a2dc9658f6e63a3b (DEFAULT_TABLE table=default.table_8b2d472c87014c6086b51a9946708d3a [id=b8a5ee6530364be1ae88e64a44dcc022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.566953 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80d6ff40e16d4114a2dc9658f6e63a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.568106 22047 consensus_queue.cc:260] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.574776 22047 ts_tablet_manager.cc:1428] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.014s user 0.003s sys 0.000s
I20260806 02:45:22.576027 21081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Sending DeleteTablet for 3 replicas of tablet c12fa3c68f52497a9ed0fe9398a46b98
I20260806 02:45:22.576678 21081 catalog_manager.cc:6565] ReplaceTablet: tablet c12fa3c68f52497a9ed0fe9398a46b98 deleted and replaced by tablet fde40ab726344932997271f125940c61
I20260806 02:45:22.577678 22047 tablet_bootstrap.cc:492] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:22.578284 21081 master_service.cc:946] ReplaceTablet: received request to replace tablet a12d9a1b4ebc496a86720ee06efd9c4e from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:22.579411 22047 tablet_bootstrap.cc:654] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.581362 22045 raft_consensus.cc:348] T a12d9a1b4ebc496a86720ee06efd9c4e P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.581771 21495 tablet_service.cc:1514] Processing DeleteTablet for tablet c12fa3c68f52497a9ed0fe9398a46b98 with delete_type TABLET_DATA_DELETED (replaced by tablet fde40ab726344932997271f125940c61) from {username='slave'} at 127.0.0.1:33104
I20260806 02:45:22.581944 22045 raft_consensus.cc:374] T a12d9a1b4ebc496a86720ee06efd9c4e P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.582065 22045 raft_consensus.cc:729] T a12d9a1b4ebc496a86720ee06efd9c4e P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
W20260806 02:45:22.582120 21064 catalog_manager.cc:4821] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): delete failed for tablet a12d9a1b4ebc496a86720ee06efd9c4e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a12d9a1b4ebc496a86720ee06efd9c4e already in progress: creating tablet
I20260806 02:45:22.582993 21865 tablet_service.cc:1514] Processing DeleteTablet for tablet c12fa3c68f52497a9ed0fe9398a46b98 with delete_type TABLET_DATA_DELETED (replaced by tablet fde40ab726344932997271f125940c61) from {username='slave'} at 127.0.0.1:56372
W20260806 02:45:22.583834 21064 catalog_manager.cc:4821] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): delete failed for tablet c12fa3c68f52497a9ed0fe9398a46b98 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c12fa3c68f52497a9ed0fe9398a46b98 already in progress: creating tablet
I20260806 02:45:22.583662 22046 raft_consensus.cc:348] T c12fa3c68f52497a9ed0fe9398a46b98 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.584208 22046 raft_consensus.cc:374] T c12fa3c68f52497a9ed0fe9398a46b98 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260806 02:45:22.584316 21064 catalog_manager.cc:4821] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): delete failed for tablet c12fa3c68f52497a9ed0fe9398a46b98 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c12fa3c68f52497a9ed0fe9398a46b98 already in progress: creating tablet
I20260806 02:45:22.584352 22046 raft_consensus.cc:729] T c12fa3c68f52497a9ed0fe9398a46b98 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.585186 22046 consensus_queue.cc:260] T c12fa3c68f52497a9ed0fe9398a46b98 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.586721 21682 tablet_service.cc:1514] Processing DeleteTablet for tablet c12fa3c68f52497a9ed0fe9398a46b98 with delete_type TABLET_DATA_DELETED (replaced by tablet fde40ab726344932997271f125940c61) from {username='slave'} at 127.0.0.1:38404
I20260806 02:45:22.586484 22045 consensus_queue.cc:260] T a12d9a1b4ebc496a86720ee06efd9c4e P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.592226 22047 tablet_bootstrap.cc:492] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:22.592392 22047 ts_tablet_manager.cc:1397] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.007s
W20260806 02:45:22.592733 21766 tablet.cc:2369] T a12d9a1b4ebc496a86720ee06efd9c4e P 88dc3ab284a64961b1fbdb5c53d1267f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:45:22.593008 22046 ts_tablet_manager.cc:1428] T c12fa3c68f52497a9ed0fe9398a46b98 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.147s user 0.016s sys 0.000s
I20260806 02:45:22.593286 22047 raft_consensus.cc:348] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.593676 22047 raft_consensus.cc:374] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.593796 22047 raft_consensus.cc:729] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.594120 22047 consensus_queue.cc:260] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.594259 21495 tablet_service.cc:1467] Processing CreateTablet for tablet 6061f59aedec4e0a9b96239ebe6fa0df (DEFAULT_TABLE table=default.table_54a040201f184199874cbda298addd03 [id=4c9f8d8c878a4b128b71e594b536267e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.594802 21495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6061f59aedec4e0a9b96239ebe6fa0df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.598421 22047 ts_tablet_manager.cc:1428] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260806 02:45:22.599565 21865 tablet_service.cc:1467] Processing CreateTablet for tablet 6061f59aedec4e0a9b96239ebe6fa0df (DEFAULT_TABLE table=default.table_54a040201f184199874cbda298addd03 [id=4c9f8d8c878a4b128b71e594b536267e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.600076 21865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6061f59aedec4e0a9b96239ebe6fa0df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.603312 21682 tablet_service.cc:1467] Processing CreateTablet for tablet 6061f59aedec4e0a9b96239ebe6fa0df (DEFAULT_TABLE table=default.table_54a040201f184199874cbda298addd03 [id=4c9f8d8c878a4b128b71e594b536267e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.603765 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6061f59aedec4e0a9b96239ebe6fa0df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.610731 22047 tablet_bootstrap.cc:492] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:22.612996 22047 tablet_bootstrap.cc:654] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.614121 21682 tablet_service.cc:1467] Processing CreateTablet for tablet fde40ab726344932997271f125940c61 (DEFAULT_TABLE table=default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.614537 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fde40ab726344932997271f125940c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.615540 22046 tablet_bootstrap.cc:492] T a12d9a1b4ebc496a86720ee06efd9c4e P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:22.617271 22046 tablet_bootstrap.cc:654] T a12d9a1b4ebc496a86720ee06efd9c4e P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.618355 21081 master_service.cc:946] ReplaceTablet: received request to replace tablet c12fa3c68f52497a9ed0fe9398a46b98 from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:22.618986 21081 master_service.cc:946] ReplaceTablet: received request to replace tablet a12d9a1b4ebc496a86720ee06efd9c4e from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:22.619715 21081 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_ea1a38afd92a4b15bfabb80e58f8c644"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:22.621263 21865 tablet_service.cc:1467] Processing CreateTablet for tablet fde40ab726344932997271f125940c61 (DEFAULT_TABLE table=default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.621759 21865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fde40ab726344932997271f125940c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.630208 22045 ts_tablet_manager.cc:1428] T a12d9a1b4ebc496a86720ee06efd9c4e P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.138s user 0.017s sys 0.008s
I20260806 02:45:22.633526 22045 tablet_bootstrap.cc:492] T c12fa3c68f52497a9ed0fe9398a46b98 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:22.635699 22045 tablet_bootstrap.cc:654] T c12fa3c68f52497a9ed0fe9398a46b98 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.642607 22061 tablet_replica.cc:330] T a12d9a1b4ebc496a86720ee06efd9c4e P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:22.643280 22061 raft_consensus.cc:2229] T a12d9a1b4ebc496a86720ee06efd9c4e P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:22.643389 22046 tablet_bootstrap.cc:492] T a12d9a1b4ebc496a86720ee06efd9c4e P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:22.643595 22046 ts_tablet_manager.cc:1397] T a12d9a1b4ebc496a86720ee06efd9c4e P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.004s
I20260806 02:45:22.643862 22061 raft_consensus.cc:2258] T a12d9a1b4ebc496a86720ee06efd9c4e P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:22.645002 22046 raft_consensus.cc:348] T a12d9a1b4ebc496a86720ee06efd9c4e P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.645515 22046 raft_consensus.cc:374] T a12d9a1b4ebc496a86720ee06efd9c4e P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.645689 22046 raft_consensus.cc:729] T a12d9a1b4ebc496a86720ee06efd9c4e P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.646075 22046 consensus_queue.cc:260] T a12d9a1b4ebc496a86720ee06efd9c4e P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.646713 22046 ts_tablet_manager.cc:1428] T a12d9a1b4ebc496a86720ee06efd9c4e P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 02:45:22.647010 22046 tablet_bootstrap.cc:492] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:22.650348 22046 tablet_bootstrap.cc:654] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.650245 22061 ts_tablet_manager.cc:1905] T a12d9a1b4ebc496a86720ee06efd9c4e P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:22.655337 21495 tablet_service.cc:1467] Processing CreateTablet for tablet fde40ab726344932997271f125940c61 (DEFAULT_TABLE table=default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.655889 21495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fde40ab726344932997271f125940c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.658110 22045 tablet_bootstrap.cc:492] T c12fa3c68f52497a9ed0fe9398a46b98 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:22.658318 22045 ts_tablet_manager.cc:1397] T c12fa3c68f52497a9ed0fe9398a46b98 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.000s
I20260806 02:45:22.658438 21077 catalog_manager.cc:2148] created HMS catalog entry for table [id=2981a18bee6a4705ae5547688e2728f8]
I20260806 02:45:22.666909 22061 ts_tablet_manager.cc:1918] T a12d9a1b4ebc496a86720ee06efd9c4e P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:22.667107 22061 log.cc:1196] T a12d9a1b4ebc496a86720ee06efd9c4e P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/a12d9a1b4ebc496a86720ee06efd9c4e
I20260806 02:45:22.667752 22061 ts_tablet_manager.cc:1939] T a12d9a1b4ebc496a86720ee06efd9c4e P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:22.667790 22045 raft_consensus.cc:348] T c12fa3c68f52497a9ed0fe9398a46b98 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.668032 22045 raft_consensus.cc:374] T c12fa3c68f52497a9ed0fe9398a46b98 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.668128 22045 raft_consensus.cc:729] T c12fa3c68f52497a9ed0fe9398a46b98 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.668431 22045 consensus_queue.cc:260] T c12fa3c68f52497a9ed0fe9398a46b98 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.669093 22045 ts_tablet_manager.cc:1428] T c12fa3c68f52497a9ed0fe9398a46b98 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20260806 02:45:22.670315 22046 tablet_bootstrap.cc:492] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:22.670508 22046 ts_tablet_manager.cc:1397] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.000s
I20260806 02:45:22.672545 21865 tablet_service.cc:1514] Processing DeleteTablet for tablet c12fa3c68f52497a9ed0fe9398a46b98 with delete_type TABLET_DATA_DELETED (replaced by tablet fde40ab726344932997271f125940c61) from {username='slave'} at 127.0.0.1:56372
I20260806 02:45:22.672906 22058 tablet_replica.cc:330] T c12fa3c68f52497a9ed0fe9398a46b98 P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:22.673483 22058 raft_consensus.cc:2229] T c12fa3c68f52497a9ed0fe9398a46b98 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:22.673640 21495 tablet_service.cc:1514] Processing DeleteTablet for tablet a12d9a1b4ebc496a86720ee06efd9c4e with delete_type TABLET_DATA_DELETED (replaced by tablet 80d6ff40e16d4114a2dc9658f6e63a3b) from {username='slave'} at 127.0.0.1:33104
I20260806 02:45:22.673998 22058 raft_consensus.cc:2258] T c12fa3c68f52497a9ed0fe9398a46b98 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:22.674711 22046 raft_consensus.cc:348] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.675019 22046 raft_consensus.cc:374] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.675104 22046 raft_consensus.cc:729] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.675343 22046 consensus_queue.cc:260] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.677134 22045 tablet_bootstrap.cc:492] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:22.677304 22057 tablet_replica.cc:330] T a12d9a1b4ebc496a86720ee06efd9c4e P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:22.677778 22057 raft_consensus.cc:2229] T a12d9a1b4ebc496a86720ee06efd9c4e P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:22.678282 22057 raft_consensus.cc:2258] T a12d9a1b4ebc496a86720ee06efd9c4e P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:22.678979 22045 tablet_bootstrap.cc:654] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.675904 22046 ts_tablet_manager.cc:1428] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260806 02:45:22.680377 22046 tablet_bootstrap.cc:492] T 043839a5eca143d5882f40faaab39692 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:22.683077 22046 tablet_bootstrap.cc:654] T 043839a5eca143d5882f40faaab39692 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.684439 22057 ts_tablet_manager.cc:1905] T a12d9a1b4ebc496a86720ee06efd9c4e P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:22.675931 22058 ts_tablet_manager.cc:1905] T c12fa3c68f52497a9ed0fe9398a46b98 P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:22.686641 21682 tablet_service.cc:1467] Processing CreateTablet for tablet cba792f10b234bc9aa7b3189fbbf1ea4 (DEFAULT_TABLE table=default.table_c94f1c901de74f71bc21408cd63fa3ea [id=2981a18bee6a4705ae5547688e2728f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.687232 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cba792f10b234bc9aa7b3189fbbf1ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.689417 21865 tablet_service.cc:1467] Processing CreateTablet for tablet cba792f10b234bc9aa7b3189fbbf1ea4 (DEFAULT_TABLE table=default.table_c94f1c901de74f71bc21408cd63fa3ea [id=2981a18bee6a4705ae5547688e2728f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.690121 21865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cba792f10b234bc9aa7b3189fbbf1ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.692741 21495 tablet_service.cc:1467] Processing CreateTablet for tablet cba792f10b234bc9aa7b3189fbbf1ea4 (DEFAULT_TABLE table=default.table_c94f1c901de74f71bc21408cd63fa3ea [id=2981a18bee6a4705ae5547688e2728f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.693198 21495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cba792f10b234bc9aa7b3189fbbf1ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.698397 21863 tablet_service.cc:1514] Processing DeleteTablet for tablet a12d9a1b4ebc496a86720ee06efd9c4e with delete_type TABLET_DATA_DELETED (replaced by tablet 80d6ff40e16d4114a2dc9658f6e63a3b) from {username='slave'} at 127.0.0.1:56372
I20260806 02:45:22.702245 21495 tablet_service.cc:1514] Processing DeleteTablet for tablet c12fa3c68f52497a9ed0fe9398a46b98 with delete_type TABLET_DATA_DELETED (replaced by tablet fde40ab726344932997271f125940c61) from {username='slave'} at 127.0.0.1:33104
I20260806 02:45:22.703126 22045 tablet_bootstrap.cc:492] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:22.704015 22046 tablet_bootstrap.cc:492] T 043839a5eca143d5882f40faaab39692 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:22.704314 22045 ts_tablet_manager.cc:1397] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.027s user 0.007s sys 0.001s
I20260806 02:45:22.705418 22045 raft_consensus.cc:348] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.707070 22046 ts_tablet_manager.cc:1397] T 043839a5eca143d5882f40faaab39692 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.009s
I20260806 02:45:22.707125 21077 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58094:
table { table_name: "default.Table_54a040201f184199874cbda298addd03" } new_table_name: "default.Table_ddb47e155c91465f8e38147105f2a6bd" modify_external_catalogs: true
I20260806 02:45:22.709149 22045 raft_consensus.cc:374] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.709426 22045 raft_consensus.cc:729] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.711272 22045 consensus_queue.cc:260] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.708155 22046 raft_consensus.cc:348] T 043839a5eca143d5882f40faaab39692 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.712419 22058 ts_tablet_manager.cc:1918] T c12fa3c68f52497a9ed0fe9398a46b98 P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:22.712469 22046 raft_consensus.cc:374] T 043839a5eca143d5882f40faaab39692 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.713557 22046 raft_consensus.cc:729] T 043839a5eca143d5882f40faaab39692 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.713614 22047 tablet_bootstrap.cc:492] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:22.713775 22047 ts_tablet_manager.cc:1397] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.103s user 0.020s sys 0.014s
I20260806 02:45:22.714082 22046 consensus_queue.cc:260] T 043839a5eca143d5882f40faaab39692 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.714562 22047 raft_consensus.cc:348] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.714823 22047 raft_consensus.cc:374] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.714902 22047 raft_consensus.cc:729] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.715672 22045 ts_tablet_manager.cc:1428] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260806 02:45:22.712596 22058 log.cc:1196] T c12fa3c68f52497a9ed0fe9398a46b98 P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/c12fa3c68f52497a9ed0fe9398a46b98
I20260806 02:45:22.717793 22045 tablet_bootstrap.cc:492] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:22.717859 22058 ts_tablet_manager.cc:1939] T c12fa3c68f52497a9ed0fe9398a46b98 P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:22.718288 22046 ts_tablet_manager.cc:1428] T 043839a5eca143d5882f40faaab39692 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.011s user 0.000s sys 0.002s
I20260806 02:45:22.717305 22057 ts_tablet_manager.cc:1918] T a12d9a1b4ebc496a86720ee06efd9c4e P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:22.718794 22057 log.cc:1196] T a12d9a1b4ebc496a86720ee06efd9c4e P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/a12d9a1b4ebc496a86720ee06efd9c4e
I20260806 02:45:22.719264 22057 ts_tablet_manager.cc:1939] T a12d9a1b4ebc496a86720ee06efd9c4e P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:22.721298 22045 tablet_bootstrap.cc:654] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.721329 21064 catalog_manager.cc:4841] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): tablet a12d9a1b4ebc496a86720ee06efd9c4e (table default.table_8b2d472c87014c6086b51a9946708d3a [id=b8a5ee6530364be1ae88e64a44dcc022]) successfully deleted
I20260806 02:45:22.721494 22047 consensus_queue.cc:260] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.722126 22047 ts_tablet_manager.cc:1428] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260806 02:45:22.722446 22047 tablet_bootstrap.cc:492] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
W20260806 02:45:22.729281 21953 tablet.cc:2369] T 043839a5eca143d5882f40faaab39692 P 8ebd96b903b944f3b06851324b737bd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:45:22.730708 22061 tablet_replica.cc:330] T c12fa3c68f52497a9ed0fe9398a46b98 P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:22.730792 21068 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet a12d9a1b4ebc496a86720ee06efd9c4e (table default.table_8b2d472c87014c6086b51a9946708d3a [id=b8a5ee6530364be1ae88e64a44dcc022]) successfully deleted
I20260806 02:45:22.730930 22061 raft_consensus.cc:2229] T c12fa3c68f52497a9ed0fe9398a46b98 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:22.731204 22061 raft_consensus.cc:2258] T c12fa3c68f52497a9ed0fe9398a46b98 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:22.732102 22057 tablet_replica.cc:330] T c12fa3c68f52497a9ed0fe9398a46b98 P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:22.732935 22046 tablet_bootstrap.cc:492] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:22.733531 22057 raft_consensus.cc:2229] T c12fa3c68f52497a9ed0fe9398a46b98 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:22.733749 22057 raft_consensus.cc:2258] T c12fa3c68f52497a9ed0fe9398a46b98 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:22.734081 22047 tablet_bootstrap.cc:654] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.734809 22046 tablet_bootstrap.cc:654] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.736198 22057 ts_tablet_manager.cc:1905] T c12fa3c68f52497a9ed0fe9398a46b98 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:22.737231 22058 tablet_replica.cc:330] T a12d9a1b4ebc496a86720ee06efd9c4e P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:22.737242 21064 catalog_manager.cc:4841] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): tablet c12fa3c68f52497a9ed0fe9398a46b98 (table default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443]) successfully deleted
I20260806 02:45:22.737458 22058 raft_consensus.cc:2229] T a12d9a1b4ebc496a86720ee06efd9c4e P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:22.737675 22058 raft_consensus.cc:2258] T a12d9a1b4ebc496a86720ee06efd9c4e P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:22.738415 22058 ts_tablet_manager.cc:1905] T a12d9a1b4ebc496a86720ee06efd9c4e P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:22.739313 22045 tablet_bootstrap.cc:492] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:22.739555 22045 ts_tablet_manager.cc:1397] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.006s
I20260806 02:45:22.742249 22045 raft_consensus.cc:348] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.742642 22045 raft_consensus.cc:374] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.742817 22045 raft_consensus.cc:729] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.742980 22061 ts_tablet_manager.cc:1905] T c12fa3c68f52497a9ed0fe9398a46b98 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:22.743232 22045 consensus_queue.cc:260] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.747331 22045 ts_tablet_manager.cc:1428] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260806 02:45:22.750613 22045 tablet_bootstrap.cc:492] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:22.752523 22045 tablet_bootstrap.cc:654] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
W20260806 02:45:22.753549 21589 tablet.cc:2369] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:45:22.755860 22046 tablet_bootstrap.cc:492] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:22.756135 22046 ts_tablet_manager.cc:1397] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.005s
I20260806 02:45:22.757066 22046 raft_consensus.cc:348] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.757498 22046 raft_consensus.cc:374] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.758440 22047 tablet_bootstrap.cc:492] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:22.758630 22046 raft_consensus.cc:729] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.759107 22046 consensus_queue.cc:260] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.759526 22047 ts_tablet_manager.cc:1397] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.037s user 0.003s sys 0.001s
I20260806 02:45:22.761392 22058 ts_tablet_manager.cc:1918] T a12d9a1b4ebc496a86720ee06efd9c4e P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:22.763530 21971 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:22.763792 21971 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:22.765085 22058 log.cc:1196] T a12d9a1b4ebc496a86720ee06efd9c4e P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/a12d9a1b4ebc496a86720ee06efd9c4e
W20260806 02:45:22.765257 21064 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111) [suppressed 9 similar messages]
I20260806 02:45:22.765707 22058 ts_tablet_manager.cc:1939] T a12d9a1b4ebc496a86720ee06efd9c4e P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:22.766981 21064 catalog_manager.cc:4841] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): tablet a12d9a1b4ebc496a86720ee06efd9c4e (table default.table_8b2d472c87014c6086b51a9946708d3a [id=b8a5ee6530364be1ae88e64a44dcc022]) successfully deleted
I20260806 02:45:22.769063 22046 ts_tablet_manager.cc:1428] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20260806 02:45:22.763062 22061 ts_tablet_manager.cc:1918] T c12fa3c68f52497a9ed0fe9398a46b98 P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:22.769366 22061 log.cc:1196] T c12fa3c68f52497a9ed0fe9398a46b98 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/c12fa3c68f52497a9ed0fe9398a46b98
I20260806 02:45:22.769855 22061 ts_tablet_manager.cc:1939] T c12fa3c68f52497a9ed0fe9398a46b98 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:22.770179 22047 raft_consensus.cc:348] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.770936 21068 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet c12fa3c68f52497a9ed0fe9398a46b98 (table default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443]) successfully deleted
I20260806 02:45:22.772620 22046 tablet_bootstrap.cc:492] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:22.773419 22047 raft_consensus.cc:374] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.774216 22045 tablet_bootstrap.cc:492] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:22.774562 22045 ts_tablet_manager.cc:1397] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.024s user 0.014s sys 0.004s
I20260806 02:45:22.774689 22046 tablet_bootstrap.cc:654] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.774840 22047 raft_consensus.cc:729] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.775177 22047 consensus_queue.cc:260] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.775548 22045 raft_consensus.cc:348] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.775882 22045 raft_consensus.cc:374] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.776029 22045 raft_consensus.cc:729] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.776360 22045 consensus_queue.cc:260] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.778813 21076 catalog_manager.cc:2148] created HMS catalog entry for table [id=21d847eb419e4c0998a01f87c8cd92da]
I20260806 02:45:22.778880 22045 ts_tablet_manager.cc:1428] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260806 02:45:22.779315 22045 tablet_bootstrap.cc:492] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:22.780510 22047 ts_tablet_manager.cc:1428] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.011s user 0.000s sys 0.002s
I20260806 02:45:22.781193 22045 tablet_bootstrap.cc:654] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.783419 22047 tablet_bootstrap.cc:492] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:22.787809 22047 tablet_bootstrap.cc:654] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.790741 22057 ts_tablet_manager.cc:1918] T c12fa3c68f52497a9ed0fe9398a46b98 P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:22.792037 22057 log.cc:1196] T c12fa3c68f52497a9ed0fe9398a46b98 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/c12fa3c68f52497a9ed0fe9398a46b98
I20260806 02:45:22.790900 22045 tablet_bootstrap.cc:492] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:22.792358 22045 ts_tablet_manager.cc:1397] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.004s
I20260806 02:45:22.792816 22046 tablet_bootstrap.cc:492] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:22.792958 22046 ts_tablet_manager.cc:1397] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20260806 02:45:22.793286 22045 raft_consensus.cc:348] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.793581 22045 raft_consensus.cc:374] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.793695 22045 raft_consensus.cc:729] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.793781 22046 raft_consensus.cc:348] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.794092 22046 raft_consensus.cc:374] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.794117 22045 consensus_queue.cc:260] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.794489 22046 raft_consensus.cc:729] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.794874 22045 ts_tablet_manager.cc:1428] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 02:45:22.794822 22046 consensus_queue.cc:260] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.795202 22045 tablet_bootstrap.cc:492] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:22.797194 22045 tablet_bootstrap.cc:654] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.800352 22046 ts_tablet_manager.cc:1428] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260806 02:45:22.803539 22046 tablet_bootstrap.cc:492] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:22.804668 22045 tablet_bootstrap.cc:492] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:22.804874 22045 ts_tablet_manager.cc:1397] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260806 02:45:22.805313 22057 ts_tablet_manager.cc:1939] T c12fa3c68f52497a9ed0fe9398a46b98 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:22.805958 22045 raft_consensus.cc:348] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.806262 22045 raft_consensus.cc:374] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.806406 22045 raft_consensus.cc:729] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.806939 22046 tablet_bootstrap.cc:654] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.806736 22045 consensus_queue.cc:260] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.807674 21064 catalog_manager.cc:4841] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): tablet c12fa3c68f52497a9ed0fe9398a46b98 (table default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443]) successfully deleted
I20260806 02:45:22.809664 22045 ts_tablet_manager.cc:1428] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260806 02:45:22.810186 22047 tablet_bootstrap.cc:492] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:22.810431 22047 ts_tablet_manager.cc:1397] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.002s
I20260806 02:45:22.811408 22047 raft_consensus.cc:348] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.811806 22045 tablet_bootstrap.cc:492] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:22.811774 22047 raft_consensus.cc:374] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.811991 22047 raft_consensus.cc:729] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.812386 22047 consensus_queue.cc:260] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.813571 22045 tablet_bootstrap.cc:654] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.816387 22047 ts_tablet_manager.cc:1428] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260806 02:45:22.819533 22047 tablet_bootstrap.cc:492] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:22.821396 21865 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0c5aff461940d999d0ed14292db320 (DEFAULT_TABLE table=default.table_a7715d8e001f48b2899c3c2a56a18a06 [id=21d847eb419e4c0998a01f87c8cd92da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.821889 21865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0c5aff461940d999d0ed14292db320. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.822072 21495 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0c5aff461940d999d0ed14292db320 (DEFAULT_TABLE table=default.table_a7715d8e001f48b2899c3c2a56a18a06 [id=21d847eb419e4c0998a01f87c8cd92da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.822587 21495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0c5aff461940d999d0ed14292db320. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.828550 21682 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0c5aff461940d999d0ed14292db320 (DEFAULT_TABLE table=default.table_a7715d8e001f48b2899c3c2a56a18a06 [id=21d847eb419e4c0998a01f87c8cd92da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.829028 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0c5aff461940d999d0ed14292db320. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.829918 21079 catalog_manager.cc:2148] created HMS catalog entry for table [id=ccf2db5226644b498abe88bba95d6fef]
I20260806 02:45:22.837806 22047 tablet_bootstrap.cc:654] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.849725 21079 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:58094 (request call id 6) took 1019 ms. Trace:
I20260806 02:45:22.850035 21079 rpcz_store.cc:276] 0806 02:45:21.830309 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:21.830395 (+ 86us) service_pool.cc:227] Handling call
0806 02:45:22.314819 (+484424us) catalog_manager.cc:2075] Acquired catalog manager lock
0806 02:45:22.315303 (+ 484us) catalog_manager.cc:2121] Created new table and tablet info
0806 02:45:22.829861 (+514558us) catalog_manager.cc:2147] Created new table in HMS catalog
0806 02:45:22.849439 (+ 19578us) catalog_manager.cc:2181] Wrote table and tablets to system table
0806 02:45:22.849563 (+ 124us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0806 02:45:22.849675 (+ 112us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0806 02:45:22.836024 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0806 02:45:22.836187 (+ 163us) write_op.cc:432] Acquiring schema lock in shared mode
0806 02:45:22.836197 (+ 10us) write_op.cc:435] Acquired schema lock
0806 02:45:22.836202 (+ 5us) tablet.cc:660] Decoding operations
0806 02:45:22.836346 (+ 144us) write_op.cc:620] Acquiring the partition lock for write op
0806 02:45:22.836366 (+ 20us) write_op.cc:641] Partition lock acquired for write op
0806 02:45:22.836374 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0806 02:45:22.836512 (+ 138us) tablet.cc:699] Row locks acquired
0806 02:45:22.836517 (+ 5us) write_op.cc:260] PREPARE: finished
0806 02:45:22.836567 (+ 50us) write_op.cc:270] Start()
0806 02:45:22.836619 (+ 52us) write_op.cc:276] Timestamp: P: 1785984322836530 usec, L: 0
0806 02:45:22.836625 (+ 6us) op_driver.cc:331] REPLICATION: starting
0806 02:45:22.836923 (+ 298us) log.cc:842] Serialized 432 byte log entry
0806 02:45:22.848604 (+ 11681us) op_driver.cc:444] REPLICATION: finished
0806 02:45:22.848782 (+ 178us) write_op.cc:301] APPLY: starting
0806 02:45:22.848817 (+ 35us) tablet.cc:1365] starting BulkCheckPresence
0806 02:45:22.848863 (+ 46us) tablet.cc:1368] finished BulkCheckPresence
0806 02:45:22.848890 (+ 27us) tablet.cc:1370] starting ApplyRowOperation cycle
0806 02:45:22.849023 (+ 133us) tablet.cc:1381] finished ApplyRowOperation cycle
0806 02:45:22.849061 (+ 38us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0806 02:45:22.849068 (+ 7us) write_op.cc:312] APPLY: finished
0806 02:45:22.849154 (+ 86us) log.cc:842] Serialized 28 byte log entry
0806 02:45:22.849222 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:22.849244 (+ 22us) write_op.cc:454] Released schema lock
0806 02:45:22.849252 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0806 02:45:22.849315 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:22.849319 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":463298,"Hive Metastore.run_cpu_time_us":289,"Hive Metastore.run_wall_time_us":50874,"child_traces":[["op",{"apply.queue_time_us":121,"num_ops":2,"prepare.queue_time_us":5379,"prepare.run_cpu_time_us":1058,"prepare.run_wall_time_us":1058,"raft.queue_time_us":752,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":7,"replication_time_us":11951}]]}
I20260806 02:45:22.853063 22047 tablet_bootstrap.cc:492] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:22.853209 22047 ts_tablet_manager.cc:1397] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.001s
I20260806 02:45:22.854421 22052 raft_consensus.cc:482] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:22.854779 22052 raft_consensus.cc:504] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.855144 22047 raft_consensus.cc:348] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.855386 22047 raft_consensus.cc:374] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.855469 22047 raft_consensus.cc:729] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.857322 22052 leader_election.cc:288] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:22.857537 22047 consensus_queue.cc:260] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.859335 22047 ts_tablet_manager.cc:1428] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260806 02:45:22.872200 20720 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
/tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.61:36797
--webserver_interface=127.20.60.61
--webserver_port=46303
--builtin_ntp_servers=127.20.60.20:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260806 02:45:22.875692 21498 tablet_service.cc:1467] Processing CreateTablet for tablet 3b3a4c79185241a7a8522135f0a513b3 (DEFAULT_TABLE table=default.table_c2d5cb4cc8a44eaca15ba851ecc366c8 [id=ccf2db5226644b498abe88bba95d6fef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.876209 21498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b3a4c79185241a7a8522135f0a513b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.882596 21865 tablet_service.cc:1467] Processing CreateTablet for tablet 3b3a4c79185241a7a8522135f0a513b3 (DEFAULT_TABLE table=default.table_c2d5cb4cc8a44eaca15ba851ecc366c8 [id=ccf2db5226644b498abe88bba95d6fef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.883157 21865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b3a4c79185241a7a8522135f0a513b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.888743 21076 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_2b3ca9846e244fc593dc07fd72a58f3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:22.895927 21683 tablet_service.cc:1467] Processing CreateTablet for tablet 3b3a4c79185241a7a8522135f0a513b3 (DEFAULT_TABLE table=default.table_c2d5cb4cc8a44eaca15ba851ecc366c8 [id=ccf2db5226644b498abe88bba95d6fef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.896211 21074 catalog_manager.cc:2148] created HMS catalog entry for table [id=f9ea378b10f94d99a856cf000c426373]
I20260806 02:45:22.896554 21683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b3a4c79185241a7a8522135f0a513b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.902776 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0956f9352c84a069760fa70da9e8881" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:22.903390 21888 raft_consensus.cc:2454] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:22.908746 21074 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:58094 (request call id 9) took 1072 ms. Trace:
I20260806 02:45:22.908902 21074 rpcz_store.cc:276] 0806 02:45:21.836433 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:21.836499 (+ 66us) service_pool.cc:227] Handling call
0806 02:45:22.315764 (+479265us) catalog_manager.cc:2075] Acquired catalog manager lock
0806 02:45:22.316188 (+ 424us) catalog_manager.cc:2121] Created new table and tablet info
0806 02:45:22.896158 (+579970us) catalog_manager.cc:2147] Created new table in HMS catalog
0806 02:45:22.908429 (+ 12271us) catalog_manager.cc:2181] Wrote table and tablets to system table
0806 02:45:22.908573 (+ 144us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0806 02:45:22.908699 (+ 126us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0806 02:45:22.896883 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0806 02:45:22.896988 (+ 105us) write_op.cc:432] Acquiring schema lock in shared mode
0806 02:45:22.896995 (+ 7us) write_op.cc:435] Acquired schema lock
0806 02:45:22.896997 (+ 2us) tablet.cc:660] Decoding operations
0806 02:45:22.897095 (+ 98us) write_op.cc:620] Acquiring the partition lock for write op
0806 02:45:22.897110 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0806 02:45:22.897116 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0806 02:45:22.897214 (+ 98us) tablet.cc:699] Row locks acquired
0806 02:45:22.897217 (+ 3us) write_op.cc:260] PREPARE: finished
0806 02:45:22.897263 (+ 46us) write_op.cc:270] Start()
0806 02:45:22.897301 (+ 38us) write_op.cc:276] Timestamp: P: 1785984322897223 usec, L: 0
0806 02:45:22.897305 (+ 4us) op_driver.cc:331] REPLICATION: starting
0806 02:45:22.897543 (+ 238us) log.cc:842] Serialized 432 byte log entry
0806 02:45:22.904559 (+ 7016us) op_driver.cc:444] REPLICATION: finished
0806 02:45:22.904968 (+ 409us) write_op.cc:301] APPLY: starting
0806 02:45:22.904988 (+ 20us) tablet.cc:1365] starting BulkCheckPresence
0806 02:45:22.905037 (+ 49us) tablet.cc:1368] finished BulkCheckPresence
0806 02:45:22.905063 (+ 26us) tablet.cc:1370] starting ApplyRowOperation cycle
0806 02:45:22.905180 (+ 117us) tablet.cc:1381] finished ApplyRowOperation cycle
0806 02:45:22.905221 (+ 41us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0806 02:45:22.905227 (+ 6us) write_op.cc:312] APPLY: finished
0806 02:45:22.905321 (+ 94us) log.cc:842] Serialized 28 byte log entry
0806 02:45:22.908289 (+ 2968us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:22.908316 (+ 27us) write_op.cc:454] Released schema lock
0806 02:45:22.908326 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0806 02:45:22.908409 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:22.908415 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":513334,"Hive Metastore.run_cpu_time_us":277,"Hive Metastore.run_wall_time_us":66279,"child_traces":[["op",{"apply.queue_time_us":343,"apply.run_cpu_time_us":663,"apply.run_wall_time_us":3527,"num_ops":2,"prepare.queue_time_us":54,"prepare.run_cpu_time_us":811,"prepare.run_wall_time_us":3889,"raft.queue_time_us":70,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":249,"replication_time_us":7225}]]}
I20260806 02:45:22.910177 21074 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_00c06d2b7d6d40658a4390f4ee3bbab5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:22.911692 22047 tablet_bootstrap.cc:492] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:22.912738 21521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0956f9352c84a069760fa70da9e8881" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:22.913394 21521 raft_consensus.cc:2454] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:22.919517 21617 leader_election.cc:302] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:22.920269 22052 raft_consensus.cc:2781] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:22.920563 22052 raft_consensus.cc:482] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:22.920781 22052 raft_consensus.cc:3037] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:22.921252 22047 tablet_bootstrap.cc:654] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.923849 22047 tablet_bootstrap.cc:492] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:22.924043 22047 ts_tablet_manager.cc:1397] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260806 02:45:22.925020 22047 raft_consensus.cc:348] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.925127 22052 raft_consensus.cc:504] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.925360 22047 raft_consensus.cc:374] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.925570 22047 raft_consensus.cc:729] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.925889 22047 consensus_queue.cc:260] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.926357 22052 leader_election.cc:288] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:22.931300 21079 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58094:
table { table_name: "default.Table_c94f1c901de74f71bc21408cd63fa3ea" } modify_external_catalogs: true
I20260806 02:45:22.932126 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0956f9352c84a069760fa70da9e8881" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:22.932401 21888 raft_consensus.cc:3037] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:22.939155 21079 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58094:
table { table_name: "default.Table_c94f1c901de74f71bc21408cd63fa3ea" } modify_external_catalogs: true
I20260806 02:45:22.950474 21521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0956f9352c84a069760fa70da9e8881" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:22.950810 21521 raft_consensus.cc:3037] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:22.952625 22046 tablet_bootstrap.cc:492] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:22.952696 21888 raft_consensus.cc:2454] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:22.952982 22046 ts_tablet_manager.cc:1397] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.149s user 0.026s sys 0.018s
I20260806 02:45:22.953421 21617 leader_election.cc:302] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:22.953833 22093 raft_consensus.cc:2781] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:22.954087 22093 raft_consensus.cc:686] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Becoming Leader. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Running, Role: LEADER
I20260806 02:45:22.954337 22046 raft_consensus.cc:348] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.954715 22046 raft_consensus.cc:374] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:22.954869 22046 raft_consensus.cc:729] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:22.954653 22093 consensus_queue.cc:237] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:22.955250 22046 consensus_queue.cc:260] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:22.955340 21865 tablet_service.cc:1467] Processing CreateTablet for tablet d8c3b721c7bd4c9b81ef7744c1230599 (DEFAULT_TABLE table=default.table_f36563aa9b94423ebebd9741529ba14c [id=f9ea378b10f94d99a856cf000c426373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.955948 21865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8c3b721c7bd4c9b81ef7744c1230599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.960376 21073 catalog_manager.cc:2148] created HMS catalog entry for table [id=f5342be18d4e43b480c14cf440e6c66e]
I20260806 02:45:22.962054 22046 ts_tablet_manager.cc:1428] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260806 02:45:22.965720 22046 tablet_bootstrap.cc:492] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:22.967259 21073 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:58094 (request call id 10) took 1128 ms. Trace:
I20260806 02:45:22.967413 21073 rpcz_store.cc:276] 0806 02:45:21.838319 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:21.838397 (+ 78us) service_pool.cc:227] Handling call
0806 02:45:22.316647 (+478250us) catalog_manager.cc:2075] Acquired catalog manager lock
0806 02:45:22.317113 (+ 466us) catalog_manager.cc:2121] Created new table and tablet info
0806 02:45:22.960318 (+643205us) catalog_manager.cc:2147] Created new table in HMS catalog
0806 02:45:22.966970 (+ 6652us) catalog_manager.cc:2181] Wrote table and tablets to system table
0806 02:45:22.967103 (+ 133us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0806 02:45:22.967210 (+ 107us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0806 02:45:22.961225 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0806 02:45:22.961416 (+ 191us) write_op.cc:432] Acquiring schema lock in shared mode
0806 02:45:22.961457 (+ 41us) write_op.cc:435] Acquired schema lock
0806 02:45:22.961461 (+ 4us) tablet.cc:660] Decoding operations
0806 02:45:22.961604 (+ 143us) write_op.cc:620] Acquiring the partition lock for write op
0806 02:45:22.961624 (+ 20us) write_op.cc:641] Partition lock acquired for write op
0806 02:45:22.961633 (+ 9us) tablet.cc:683] Acquiring locks for 2 operations
0806 02:45:22.961755 (+ 122us) tablet.cc:699] Row locks acquired
0806 02:45:22.961760 (+ 5us) write_op.cc:260] PREPARE: finished
0806 02:45:22.961806 (+ 46us) write_op.cc:270] Start()
0806 02:45:22.961854 (+ 48us) write_op.cc:276] Timestamp: P: 1785984322961758 usec, L: 0
0806 02:45:22.961860 (+ 6us) op_driver.cc:331] REPLICATION: starting
0806 02:45:22.962102 (+ 242us) log.cc:842] Serialized 432 byte log entry
0806 02:45:22.966251 (+ 4149us) op_driver.cc:444] REPLICATION: finished
0806 02:45:22.966415 (+ 164us) write_op.cc:301] APPLY: starting
0806 02:45:22.966444 (+ 29us) tablet.cc:1365] starting BulkCheckPresence
0806 02:45:22.966494 (+ 50us) tablet.cc:1368] finished BulkCheckPresence
0806 02:45:22.966521 (+ 27us) tablet.cc:1370] starting ApplyRowOperation cycle
0806 02:45:22.966634 (+ 113us) tablet.cc:1381] finished ApplyRowOperation cycle
0806 02:45:22.966675 (+ 41us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0806 02:45:22.966682 (+ 7us) write_op.cc:312] APPLY: finished
0806 02:45:22.966767 (+ 85us) log.cc:842] Serialized 28 byte log entry
0806 02:45:22.966834 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:22.966856 (+ 22us) write_op.cc:454] Released schema lock
0806 02:45:22.966864 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0806 02:45:22.966930 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:22.966934 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":578732,"Hive Metastore.run_cpu_time_us":262,"Hive Metastore.run_wall_time_us":64195,"child_traces":[["op",{"apply.queue_time_us":99,"apply.run_cpu_time_us":571,"apply.run_wall_time_us":571,"num_ops":2,"prepare.queue_time_us":49,"prepare.run_cpu_time_us":1029,"prepare.run_wall_time_us":1029,"raft.queue_time_us":526,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":321,"replication_time_us":4366}]]}
I20260806 02:45:22.972306 21073 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_cac77bd182314ee2a2068374ddeca8d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:22.973496 21682 tablet_service.cc:1467] Processing CreateTablet for tablet d8c3b721c7bd4c9b81ef7744c1230599 (DEFAULT_TABLE table=default.table_f36563aa9b94423ebebd9741529ba14c [id=f9ea378b10f94d99a856cf000c426373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:22.974046 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8c3b721c7bd4c9b81ef7744c1230599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:22.988014 22046 tablet_bootstrap.cc:654] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:22.998499 22046 tablet_bootstrap.cc:492] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:22.998755 22046 ts_tablet_manager.cc:1397] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.008s
I20260806 02:45:22.998834 22047 ts_tablet_manager.cc:1428] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.075s user 0.005s sys 0.000s
I20260806 02:45:22.999351 22047 tablet_bootstrap.cc:492] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:22.999763 22046 raft_consensus.cc:348] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.000111 22046 raft_consensus.cc:374] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:23.000268 22046 raft_consensus.cc:729] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:23.000622 22046 consensus_queue.cc:260] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.002621 21495 tablet_service.cc:1467] Processing CreateTablet for tablet d8c3b721c7bd4c9b81ef7744c1230599 (DEFAULT_TABLE table=default.table_f36563aa9b94423ebebd9741529ba14c [id=f9ea378b10f94d99a856cf000c426373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:23.003108 21495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8c3b721c7bd4c9b81ef7744c1230599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:23.007340 21521 raft_consensus.cc:2454] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:23.009939 22045 tablet_bootstrap.cc:492] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:23.009886 21865 tablet_service.cc:1467] Processing CreateTablet for tablet d25a9c5a0123446dac00ad895381f96a (DEFAULT_TABLE table=default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:23.010130 22045 ts_tablet_manager.cc:1397] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.198s user 0.016s sys 0.078s
I20260806 02:45:23.010418 21865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25a9c5a0123446dac00ad895381f96a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:23.011725 22064 raft_consensus.cc:482] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:23.011991 22064 raft_consensus.cc:504] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.013383 22046 ts_tablet_manager.cc:1428] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.014s user 0.000s sys 0.001s
I20260806 02:45:23.014303 22064 leader_election.cc:288] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:23.015748 22045 raft_consensus.cc:348] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.016076 22045 raft_consensus.cc:374] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:23.016217 22045 raft_consensus.cc:729] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:23.016485 21495 tablet_service.cc:1467] Processing CreateTablet for tablet d25a9c5a0123446dac00ad895381f96a (DEFAULT_TABLE table=default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:23.016773 22045 consensus_queue.cc:260] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.016980 21495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25a9c5a0123446dac00ad895381f96a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:23.018486 22045 ts_tablet_manager.cc:1428] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260806 02:45:23.018854 22045 tablet_bootstrap.cc:492] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:23.023031 21682 tablet_service.cc:1467] Processing CreateTablet for tablet d25a9c5a0123446dac00ad895381f96a (DEFAULT_TABLE table=default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:23.023579 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25a9c5a0123446dac00ad895381f96a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:23.025370 22045 tablet_bootstrap.cc:654] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:23.025774 22093 raft_consensus.cc:482] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:23.025954 22093 raft_consensus.cc:504] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.026625 22093 leader_election.cc:288] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:23.027858 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "043839a5eca143d5882f40faaab39692" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:23.028128 21888 raft_consensus.cc:2454] T 043839a5eca143d5882f40faaab39692 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:23.028293 22047 tablet_bootstrap.cc:654] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:23.028667 21617 leader_election.cc:302] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:23.029091 22093 raft_consensus.cc:2781] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:23.029198 22093 raft_consensus.cc:482] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:23.029354 22093 raft_consensus.cc:3037] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.032146 22093 raft_consensus.cc:504] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.033041 22046 tablet_bootstrap.cc:492] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:23.033211 21520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "043839a5eca143d5882f40faaab39692" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:23.033525 21520 raft_consensus.cc:3037] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.035140 22046 tablet_bootstrap.cc:654] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:23.036661 21520 raft_consensus.cc:2454] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:23.037163 21617 leader_election.cc:302] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f; no voters:
I20260806 02:45:23.037539 22052 raft_consensus.cc:2781] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:23.038093 21521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "043839a5eca143d5882f40faaab39692" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:23.038358 21521 raft_consensus.cc:2362] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1. Re-sending same reply.
I20260806 02:45:23.041857 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "043839a5eca143d5882f40faaab39692" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:23.042095 21888 raft_consensus.cc:3037] T 043839a5eca143d5882f40faaab39692 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.045806 22093 leader_election.cc:288] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:23.045892 22047 tablet_bootstrap.cc:492] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:23.046088 22047 ts_tablet_manager.cc:1397] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.047s user 0.000s sys 0.007s
I20260806 02:45:23.047129 22047 raft_consensus.cc:348] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.047407 22047 raft_consensus.cc:374] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:23.047451 22046 tablet_bootstrap.cc:492] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:23.047537 22047 raft_consensus.cc:729] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:23.047748 22046 ts_tablet_manager.cc:1397] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.006s
I20260806 02:45:23.047835 22047 consensus_queue.cc:260] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.048547 22047 ts_tablet_manager.cc:1428] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 02:45:23.048907 22047 tablet_bootstrap.cc:492] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:23.048864 22046 raft_consensus.cc:348] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.049245 22046 raft_consensus.cc:374] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:23.049414 22046 raft_consensus.cc:729] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:23.049751 22046 consensus_queue.cc:260] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.051429 21888 raft_consensus.cc:2454] T 043839a5eca143d5882f40faaab39692 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:23.054355 22046 ts_tablet_manager.cc:1428] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260806 02:45:23.058496 22046 tablet_bootstrap.cc:492] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:23.059823 22047 tablet_bootstrap.cc:654] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:23.060793 22046 tablet_bootstrap.cc:654] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:23.068071 22052 raft_consensus.cc:686] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Becoming Leader. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Running, Role: LEADER
I20260806 02:45:23.068455 22052 consensus_queue.cc:237] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.069227 22046 tablet_bootstrap.cc:492] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:23.069448 22046 ts_tablet_manager.cc:1397] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.006s
I20260806 02:45:23.070478 22046 raft_consensus.cc:348] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.070822 22047 tablet_bootstrap.cc:492] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:23.070854 22046 raft_consensus.cc:374] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:23.071024 22047 ts_tablet_manager.cc:1397] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20260806 02:45:23.071040 22046 raft_consensus.cc:729] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:23.071381 22046 consensus_queue.cc:260] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.071992 22047 raft_consensus.cc:348] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.073945 22047 raft_consensus.cc:374] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:23.074040 22047 raft_consensus.cc:729] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:23.074301 22047 consensus_queue.cc:260] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.075917 22046 ts_tablet_manager.cc:1428] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260806 02:45:23.076704 22047 ts_tablet_manager.cc:1428] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260806 02:45:23.077737 22047 tablet_bootstrap.cc:492] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:23.079718 22046 tablet_bootstrap.cc:492] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:23.081455 22046 tablet_bootstrap.cc:654] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:23.085012 22047 tablet_bootstrap.cc:654] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:23.090072 22047 tablet_bootstrap.cc:492] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:23.090147 22046 tablet_bootstrap.cc:492] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:23.090277 22047 ts_tablet_manager.cc:1397] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.003s
I20260806 02:45:23.090461 22046 ts_tablet_manager.cc:1397] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.009s
I20260806 02:45:23.091221 22047 raft_consensus.cc:348] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.091552 22047 raft_consensus.cc:374] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:23.091341 22046 raft_consensus.cc:348] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.091670 22047 raft_consensus.cc:729] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:23.091835 22046 raft_consensus.cc:374] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:23.091984 22046 raft_consensus.cc:729] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:23.092338 22046 consensus_queue.cc:260] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.096536 22046 ts_tablet_manager.cc:1428] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260806 02:45:23.101056 22046 tablet_bootstrap.cc:492] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:23.102551 22045 tablet_bootstrap.cc:492] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:23.102779 22045 ts_tablet_manager.cc:1397] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.084s user 0.006s sys 0.011s
I20260806 02:45:23.103137 22046 tablet_bootstrap.cc:654] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:23.103776 22045 raft_consensus.cc:348] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.104092 22045 raft_consensus.cc:374] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:23.104195 22045 raft_consensus.cc:729] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:23.104494 22045 consensus_queue.cc:260] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.105302 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "043839a5eca143d5882f40faaab39692" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" is_pre_election: true
I20260806 02:45:23.106081 22045 ts_tablet_manager.cc:1428] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 02:45:23.106467 22045 tablet_bootstrap.cc:492] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:23.107148 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "043839a5eca143d5882f40faaab39692" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:23.107565 21888 raft_consensus.cc:2379] T 043839a5eca143d5882f40faaab39692 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in current term 1: Already voted for candidate 88dc3ab284a64961b1fbdb5c53d1267f in this term.
I20260806 02:45:23.110652 21436 leader_election.cc:302] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:45:23.111147 22064 raft_consensus.cc:2726] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:45:23.111548 22045 tablet_bootstrap.cc:654] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:23.111687 22047 consensus_queue.cc:260] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.112466 22047 ts_tablet_manager.cc:1428] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.022s user 0.000s sys 0.002s
I20260806 02:45:23.116586 22064 raft_consensus.cc:482] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:23.116767 22064 raft_consensus.cc:504] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.117545 22064 leader_election.cc:288] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:23.118094 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0c5aff461940d999d0ed14292db320" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" is_pre_election: true
I20260806 02:45:23.118443 21710 raft_consensus.cc:2454] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 0.
I20260806 02:45:23.118749 22046 tablet_bootstrap.cc:492] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:23.118819 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0c5aff461940d999d0ed14292db320" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:23.118963 21436 leader_election.cc:302] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f; no voters:
I20260806 02:45:23.119040 22046 ts_tablet_manager.cc:1397] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.008s
I20260806 02:45:23.119236 22064 raft_consensus.cc:2781] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:23.119421 22064 raft_consensus.cc:482] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:23.119540 22064 raft_consensus.cc:3037] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.119038 21888 raft_consensus.cc:2454] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 0.
I20260806 02:45:23.120234 22046 raft_consensus.cc:348] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.120534 22046 raft_consensus.cc:374] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:23.120627 22046 raft_consensus.cc:729] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:23.120863 22046 consensus_queue.cc:260] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.122673 22046 ts_tablet_manager.cc:1428] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 02:45:23.122560 22064 raft_consensus.cc:504] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.123319 22064 leader_election.cc:288] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 election: Requested vote from peers 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:23.123873 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0c5aff461940d999d0ed14292db320" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
I20260806 02:45:23.123950 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0c5aff461940d999d0ed14292db320" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:23.124137 21888 raft_consensus.cc:3037] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.124153 21710 raft_consensus.cc:3037] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.126958 21888 raft_consensus.cc:2454] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 1.
I20260806 02:45:23.127272 21710 raft_consensus.cc:2454] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 1.
I20260806 02:45:23.127358 21434 leader_election.cc:302] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:23.127858 22064 raft_consensus.cc:2781] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:23.128412 22064 raft_consensus.cc:686] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 LEADER]: Becoming Leader. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Running, Role: LEADER
I20260806 02:45:23.129395 22064 consensus_queue.cc:237] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.132377 22045 tablet_bootstrap.cc:492] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:23.132624 22045 ts_tablet_manager.cc:1397] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.003s
I20260806 02:45:23.133749 22045 raft_consensus.cc:348] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.134212 22045 raft_consensus.cc:374] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:23.134390 22045 raft_consensus.cc:729] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:23.134766 22045 consensus_queue.cc:260] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.135633 22045 ts_tablet_manager.cc:1428] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 02:45:23.136099 22045 tablet_bootstrap.cc:492] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:23.138211 22045 tablet_bootstrap.cc:654] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:23.158816 22045 tablet_bootstrap.cc:492] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:23.159301 22045 ts_tablet_manager.cc:1397] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.017s
I20260806 02:45:23.160630 22045 raft_consensus.cc:348] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.161039 22045 raft_consensus.cc:374] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:23.161218 22045 raft_consensus.cc:729] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:23.161590 22045 consensus_queue.cc:260] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.162485 22045 ts_tablet_manager.cc:1428] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 02:45:23.162878 22045 tablet_bootstrap.cc:492] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:23.164883 22045 tablet_bootstrap.cc:654] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:23.166282 22064 raft_consensus.cc:482] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:23.166466 22064 raft_consensus.cc:504] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.167066 22064 leader_election.cc:288] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:23.167658 22052 raft_consensus.cc:482] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:23.167876 22052 raft_consensus.cc:504] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.168629 22052 leader_election.cc:288] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:23.168922 22063 raft_consensus.cc:482] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:23.170596 21521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fde40ab726344932997271f125940c61" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:23.170873 21521 raft_consensus.cc:2454] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:23.171248 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fde40ab726344932997271f125940c61" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:23.171545 21888 raft_consensus.cc:2454] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:23.172732 21617 leader_election.cc:302] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:23.173102 22052 raft_consensus.cc:2781] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:23.173218 22052 raft_consensus.cc:482] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:23.173367 22052 raft_consensus.cc:3037] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.173604 22064 raft_consensus.cc:482] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:23.173851 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fde40ab726344932997271f125940c61" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" is_pre_election: true
I20260806 02:45:23.173813 22064 raft_consensus.cc:504] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.169152 22063 raft_consensus.cc:504] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.174677 22064 leader_election.cc:288] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:23.176023 22063 leader_election.cc:288] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:23.177836 21707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6061f59aedec4e0a9b96239ebe6fa0df" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" is_pre_election: true
I20260806 02:45:23.178156 21707 raft_consensus.cc:2454] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 0.
I20260806 02:45:23.179384 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fde40ab726344932997271f125940c61" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:23.179612 21888 raft_consensus.cc:2454] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 0.
I20260806 02:45:23.180064 21436 leader_election.cc:302] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f; no voters:
I20260806 02:45:23.179952 21887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6061f59aedec4e0a9b96239ebe6fa0df" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:23.180239 21887 raft_consensus.cc:2454] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 0.
I20260806 02:45:23.180373 22064 raft_consensus.cc:2781] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:23.180554 22064 raft_consensus.cc:482] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:23.180665 22064 raft_consensus.cc:3037] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.180395 21434 leader_election.cc:302] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:23.181677 22112 raft_consensus.cc:2781] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:23.181798 22112 raft_consensus.cc:482] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:23.181946 22112 raft_consensus.cc:3037] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.184504 22064 raft_consensus.cc:504] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.184823 22112 raft_consensus.cc:504] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.185722 22112 leader_election.cc:288] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 election: Requested vote from peers 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:23.185910 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fde40ab726344932997271f125940c61" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:23.186149 21888 raft_consensus.cc:3037] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.186450 21707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fde40ab726344932997271f125940c61" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
I20260806 02:45:23.186616 21887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6061f59aedec4e0a9b96239ebe6fa0df" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:23.186800 21887 raft_consensus.cc:3037] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.189819 21887 raft_consensus.cc:2454] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 1.
I20260806 02:45:23.190026 21888 raft_consensus.cc:2454] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 1.
I20260806 02:45:23.190303 21434 leader_election.cc:302] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:23.190717 22112 raft_consensus.cc:2781] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:23.191156 21434 leader_election.cc:302] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:23.194171 22112 raft_consensus.cc:686] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 LEADER]: Becoming Leader. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Running, Role: LEADER
I20260806 02:45:23.194526 22112 consensus_queue.cc:237] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.194839 22121 raft_consensus.cc:2781] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:23.177170 22052 raft_consensus.cc:504] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.198755 21521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c3b721c7bd4c9b81ef7744c1230599" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:23.199046 21710 raft_consensus.cc:2379] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in current term 1: Already voted for candidate 88dc3ab284a64961b1fbdb5c53d1267f in this term.
I20260806 02:45:23.199036 21521 raft_consensus.cc:2454] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:23.199690 21803 leader_election.cc:302] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:23.199959 21521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fde40ab726344932997271f125940c61" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:23.200342 22063 raft_consensus.cc:2781] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:23.200500 22063 raft_consensus.cc:482] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:23.200541 22052 leader_election.cc:288] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:23.200624 22063 raft_consensus.cc:3037] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.201617 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fde40ab726344932997271f125940c61" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:23.201876 21888 raft_consensus.cc:2379] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 88dc3ab284a64961b1fbdb5c53d1267f in current term 1: Already voted for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in this term.
I20260806 02:45:23.201889 22064 leader_election.cc:288] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 election: Requested vote from peers 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:23.202877 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6061f59aedec4e0a9b96239ebe6fa0df" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
I20260806 02:45:23.203107 21710 raft_consensus.cc:3037] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.206727 21710 raft_consensus.cc:2454] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 1.
I20260806 02:45:23.209483 22045 tablet_bootstrap.cc:492] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:23.209712 22045 ts_tablet_manager.cc:1397] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.047s user 0.011s sys 0.013s
I20260806 02:45:23.210660 22045 raft_consensus.cc:348] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.211199 22045 raft_consensus.cc:374] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:23.211351 22045 raft_consensus.cc:729] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:23.211621 22045 consensus_queue.cc:260] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.212483 22045 ts_tablet_manager.cc:1428] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 02:45:23.214725 22052 raft_consensus.cc:482] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:23.214942 22052 raft_consensus.cc:504] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.215651 22052 leader_election.cc:288] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:23.216388 21520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b3a4c79185241a7a8522135f0a513b3" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:23.216492 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b3a4c79185241a7a8522135f0a513b3" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:23.216681 21520 raft_consensus.cc:2454] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:23.216739 21888 raft_consensus.cc:2454] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:23.217167 21617 leader_election.cc:302] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:23.217612 22063 raft_consensus.cc:504] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.217778 22052 raft_consensus.cc:2781] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:23.217869 22052 raft_consensus.cc:482] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:23.217994 22052 raft_consensus.cc:3037] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.218317 22063 leader_election.cc:288] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:23.218514 22063 raft_consensus.cc:482] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:23.218641 22063 raft_consensus.cc:504] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.219193 22063 leader_election.cc:288] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:23.220448 22121 raft_consensus.cc:686] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 LEADER]: Becoming Leader. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Running, Role: LEADER
I20260806 02:45:23.220785 22052 raft_consensus.cc:504] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.220831 22121 consensus_queue.cc:237] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.222208 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b3a4c79185241a7a8522135f0a513b3" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:23.222437 21888 raft_consensus.cc:3037] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.222719 21520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b3a4c79185241a7a8522135f0a513b3" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:23.222939 21520 raft_consensus.cc:3037] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.225819 21888 raft_consensus.cc:2454] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:23.226338 21617 leader_election.cc:302] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:23.226311 21519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c3b721c7bd4c9b81ef7744c1230599" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:23.226617 21519 raft_consensus.cc:3037] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.226645 22100 raft_consensus.cc:2781] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:23.226794 22100 raft_consensus.cc:686] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Becoming Leader. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Running, Role: LEADER
I20260806 02:45:23.227166 22100 consensus_queue.cc:237] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.228082 21520 raft_consensus.cc:2454] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:23.229900 21519 raft_consensus.cc:2454] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:23.230582 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c3b721c7bd4c9b81ef7744c1230599" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" is_pre_election: true
I20260806 02:45:23.230890 21710 raft_consensus.cc:2454] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:23.230731 21519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cba792f10b234bc9aa7b3189fbbf1ea4" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:23.231153 21519 raft_consensus.cc:2454] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:23.231679 21803 leader_election.cc:302] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:23.232069 22126 raft_consensus.cc:2781] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:23.232090 21803 leader_election.cc:302] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:23.232189 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c3b721c7bd4c9b81ef7744c1230599" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
I20260806 02:45:23.232297 22126 raft_consensus.cc:686] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [term 1 LEADER]: Becoming Leader. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Running, Role: LEADER
I20260806 02:45:23.232424 21710 raft_consensus.cc:3037] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.232872 22052 leader_election.cc:288] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:23.232913 22126 consensus_queue.cc:237] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.235198 21710 raft_consensus.cc:2454] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:23.236142 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cba792f10b234bc9aa7b3189fbbf1ea4" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" is_pre_election: true
I20260806 02:45:23.236371 21710 raft_consensus.cc:2454] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:23.247275 21617 leader_election.cc:302] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:45:23.247786 22052 raft_consensus.cc:2726] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 02:45:23.250325 22063 raft_consensus.cc:2781] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:23.250519 22063 raft_consensus.cc:482] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:23.250634 22063 raft_consensus.cc:3037] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.271258 22063 raft_consensus.cc:504] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.272254 22063 leader_election.cc:288] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:23.273072 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cba792f10b234bc9aa7b3189fbbf1ea4" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
I20260806 02:45:23.273370 21710 raft_consensus.cc:3037] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.276338 21710 raft_consensus.cc:2454] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:23.276976 21806 leader_election.cc:302] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:23.280398 22126 raft_consensus.cc:2781] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:23.280601 22126 raft_consensus.cc:686] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [term 1 LEADER]: Becoming Leader. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Running, Role: LEADER
I20260806 02:45:23.280925 22126 consensus_queue.cc:237] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.306612 22121 raft_consensus.cc:482] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260806 02:45:23.306361 21803 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.20.60.1:36593, user_credentials={real_user=slave}} blocked reactor thread for 74269.8us
I20260806 02:45:23.306860 22121 raft_consensus.cc:504] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:23.318457 21521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cba792f10b234bc9aa7b3189fbbf1ea4" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:23.318800 21521 raft_consensus.cc:3037] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.321620 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80d6ff40e16d4114a2dc9658f6e63a3b" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" is_pre_election: true
I20260806 02:45:23.321954 21710 raft_consensus.cc:2454] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 0.
I20260806 02:45:23.322558 21436 leader_election.cc:302] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f; no voters:
I20260806 02:45:23.322932 22112 raft_consensus.cc:2781] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:23.323053 22121 leader_election.cc:288] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:23.323052 22112 raft_consensus.cc:482] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:23.323221 22112 raft_consensus.cc:3037] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.323822 21521 raft_consensus.cc:2454] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:23.323823 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80d6ff40e16d4114a2dc9658f6e63a3b" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:23.324106 21888 raft_consensus.cc:2454] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 0.
I20260806 02:45:23.325960 22112 raft_consensus.cc:504] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:23.327471 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80d6ff40e16d4114a2dc9658f6e63a3b" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:23.327694 21888 raft_consensus.cc:3037] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.327548 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80d6ff40e16d4114a2dc9658f6e63a3b" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
I20260806 02:45:23.328032 21710 raft_consensus.cc:3037] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.330614 21888 raft_consensus.cc:2454] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 1.
I20260806 02:45:23.331133 21434 leader_election.cc:302] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:23.331514 22121 raft_consensus.cc:2781] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:23.331682 22121 raft_consensus.cc:686] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 LEADER]: Becoming Leader. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Running, Role: LEADER
I20260806 02:45:23.331943 22121 consensus_queue.cc:237] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:23.332059 21710 raft_consensus.cc:2454] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 1.
I20260806 02:45:23.334813 22112 leader_election.cc:288] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 election: Requested vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:23.438532 22052 raft_consensus.cc:482] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:23.438830 22052 raft_consensus.cc:504] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.439800 22052 leader_election.cc:288] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:23.440932 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25a9c5a0123446dac00ad895381f96a" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:23.441609 21521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25a9c5a0123446dac00ad895381f96a" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:23.441884 21521 raft_consensus.cc:2454] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:23.442621 21617 leader_election.cc:302] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f; no voters:
I20260806 02:45:23.442947 22052 raft_consensus.cc:2781] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:23.443099 22052 raft_consensus.cc:482] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:23.441275 21888 raft_consensus.cc:2454] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:23.443179 22052 raft_consensus.cc:3037] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.446548 22052 raft_consensus.cc:504] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.448122 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25a9c5a0123446dac00ad895381f96a" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:23.448397 21888 raft_consensus.cc:3037] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.452472 21521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25a9c5a0123446dac00ad895381f96a" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:23.452778 21521 raft_consensus.cc:3037] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:23.454862 22052 leader_election.cc:288] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Requested vote from peers 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:23.455834 21888 raft_consensus.cc:2454] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:23.456552 21617 leader_election.cc:302] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:23.456760 21521 raft_consensus.cc:2454] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:23.457625 22052 raft_consensus.cc:2781] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:23.459419 22052 raft_consensus.cc:686] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Becoming Leader. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Running, Role: LEADER
I20260806 02:45:23.459733 22052 consensus_queue.cc:237] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:23.513374 22135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a225e002d607451a94b3686f10aeea9a)
I20260806 02:45:23.513837 22135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:23.515797 21096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "894be089543b4aa29b3edfcc5f3085e9" candidate_term: 4 candidate_status { last_received { term: 3 index: 30 } } ignore_live_leader: false dest_uuid: "a225e002d607451a94b3686f10aeea9a" is_pre_election: true
I20260806 02:45:23.523003 22135 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
I20260806 02:45:23.525496 21082 catalog_manager.cc:3411] renamed table default.table_8b2d472c87014c6086b51a9946708d3a [id=b8a5ee6530364be1ae88e64a44dcc022] in HMS: new name default.table_e24bd493305d46a4977ed544ce47cc83
W20260806 02:45:23.531358 21081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_ea1a38afd92a4b15bfabb80e58f8c644 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 02:45:23.531350 21251 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:45:23.531559 21074 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_00c06d2b7d6d40658a4390f4ee3bbab5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 02:45:23.533021 21076 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_2b3ca9846e244fc593dc07fd72a58f3f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 02:45:23.533139 21073 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_cac77bd182314ee2a2068374ddeca8d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 02:45:23.539808 21251 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:45:23.540066 21251 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:23.540371 22135 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260806 02:45:23.606223 21391 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 80d6ff40e16d4114a2dc9658f6e63a3b consensus state has no leader
I20260806 02:45:23.609253 21082 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:58094 (request call id 11) took 1270 ms. Trace:
I20260806 02:45:23.609447 21082 rpcz_store.cc:276] 0806 02:45:22.339153 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:22.341296 (+ 2143us) service_pool.cc:227] Handling call
0806 02:45:22.992345 (+651049us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0806 02:45:23.608209 (+615864us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0806 02:45:23.608684 (+ 475us) catalog_manager.cc:3616] Apply alter schema
0806 02:45:23.609192 (+ 508us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":1327,"Hive Metastore.run_cpu_time_us":2312,"Hive Metastore.run_wall_time_us":531196}
I20260806 02:45:23.614960 21082 catalog_manager.cc:5495] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f reported cstate change: term changed from 0 to 1, leader changed from <none> to 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2). New cstate: current_term: 1 leader_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: UNKNOWN } } }
W20260806 02:45:23.636924 21068 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 80d6ff40e16d4114a2dc9658f6e63a3b (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 80d6ff40e16d4114a2dc9658f6e63a3b consensus state has no leader
I20260806 02:45:23.638885 21082 catalog_manager.cc:5495] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3). New cstate: current_term: 1 leader_uuid: "8ebd96b903b944f3b06851324b737bd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: UNKNOWN } } }
I20260806 02:45:23.640069 21082 catalog_manager.cc:5495] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3). New cstate: current_term: 1 leader_uuid: "8ebd96b903b944f3b06851324b737bd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: UNKNOWN } } }
I20260806 02:45:23.651259 21082 catalog_manager.cc:5495] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f reported cstate change: term changed from 0 to 1, leader changed from <none> to 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2). New cstate: current_term: 1 leader_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: HEALTHY } } }
I20260806 02:45:23.651844 21082 catalog_manager.cc:5495] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f reported cstate change: term changed from 0 to 1, leader changed from <none> to 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2). New cstate: current_term: 1 leader_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: HEALTHY } } }
I20260806 02:45:23.652619 21082 catalog_manager.cc:5495] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f reported cstate change: term changed from 0 to 1, leader changed from <none> to 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2). New cstate: current_term: 1 leader_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: HEALTHY } } }
I20260806 02:45:23.657033 21084 catalog_manager.cc:3411] renamed table default.table_910edc3d61454438a69af3c32d9b721f [id=1bbd1184e7754c5bb9ffdf82791c73fb] in HMS: new name default.table_e41f56f2e4a1439ca6b8f2bca460a670
I20260806 02:45:23.671241 21082 catalog_manager.cc:5495] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1). New cstate: current_term: 1 leader_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: UNKNOWN } } }
I20260806 02:45:23.672737 21082 catalog_manager.cc:5495] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1). New cstate: current_term: 1 leader_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260806 02:45:23.673750 21082 catalog_manager.cc:5587] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593) does not have the latest schema for tablet 80d6ff40e16d4114a2dc9658f6e63a3b (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]). Expected version 1 got 0
I20260806 02:45:23.674664 21082 catalog_manager.cc:5495] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1). New cstate: current_term: 1 leader_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: UNKNOWN } } }
I20260806 02:45:23.675923 21082 catalog_manager.cc:5495] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1). New cstate: current_term: 1 leader_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: UNKNOWN } } }
I20260806 02:45:23.689651 21082 master_service.cc:946] ReplaceTablet: received request to replace tablet 80d6ff40e16d4114a2dc9658f6e63a3b from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:23.706815 21082 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Sending DeleteTablet for 3 replicas of tablet 80d6ff40e16d4114a2dc9658f6e63a3b
I20260806 02:45:23.708695 21682 tablet_service.cc:1514] Processing DeleteTablet for tablet 80d6ff40e16d4114a2dc9658f6e63a3b with delete_type TABLET_DATA_DELETED (replaced by tablet 2d8219e0438e46f783037537e687f945) from {username='slave'} at 127.0.0.1:38404
I20260806 02:45:23.713888 22149 tablet_replica.cc:330] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:23.714243 22149 raft_consensus.cc:2229] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:23.714553 22149 raft_consensus.cc:2258] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:23.715315 22149 ts_tablet_manager.cc:1905] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:23.715579 21082 catalog_manager.cc:6565] ReplaceTablet: tablet 80d6ff40e16d4114a2dc9658f6e63a3b deleted and replaced by tablet 2d8219e0438e46f783037537e687f945
I20260806 02:45:23.717124 21082 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_1ff877d50df44dccb8be0241b847e8c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:23.722585 21682 tablet_service.cc:1467] Processing CreateTablet for tablet 2d8219e0438e46f783037537e687f945 (DEFAULT_TABLE table=default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:23.723486 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d8219e0438e46f783037537e687f945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:23.735600 22149 ts_tablet_manager.cc:1918] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:23.737540 22149 log.cc:1196] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/80d6ff40e16d4114a2dc9658f6e63a3b
I20260806 02:45:23.738476 22149 ts_tablet_manager.cc:1939] T 80d6ff40e16d4114a2dc9658f6e63a3b P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:23.741570 21068 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet 80d6ff40e16d4114a2dc9658f6e63a3b (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]) successfully deleted
I20260806 02:45:23.745121 22150 tablet_bootstrap.cc:492] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:23.751134 22150 tablet_bootstrap.cc:654] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:23.759202 21495 tablet_service.cc:1514] Processing DeleteTablet for tablet 80d6ff40e16d4114a2dc9658f6e63a3b with delete_type TABLET_DATA_DELETED (replaced by tablet 2d8219e0438e46f783037537e687f945) from {username='slave'} at 127.0.0.1:33104
I20260806 02:45:23.762974 22150 tablet_bootstrap.cc:492] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:23.763229 22150 ts_tablet_manager.cc:1397] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.000s
I20260806 02:45:23.778501 22150 raft_consensus.cc:348] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:23.778931 22150 raft_consensus.cc:374] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:23.779160 22150 raft_consensus.cc:729] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:23.779170 21865 tablet_service.cc:1514] Processing DeleteTablet for tablet 80d6ff40e16d4114a2dc9658f6e63a3b with delete_type TABLET_DATA_DELETED (replaced by tablet 2d8219e0438e46f783037537e687f945) from {username='slave'} at 127.0.0.1:56372
I20260806 02:45:23.779623 22150 consensus_queue.cc:260] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:23.780527 22150 ts_tablet_manager.cc:1428] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.016s user 0.002s sys 0.000s
I20260806 02:45:23.802410 21494 tablet_service.cc:1467] Processing CreateTablet for tablet 2d8219e0438e46f783037537e687f945 (DEFAULT_TABLE table=default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:23.803256 21494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d8219e0438e46f783037537e687f945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:23.805863 21083 catalog_manager.cc:3411] renamed table default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443] in HMS: new name default.table_8797f6f61c444bd0a4e4555f6ca2fe30
I20260806 02:45:23.808856 22052 consensus_queue.cc:1035] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:23.824225 21863 tablet_service.cc:1467] Processing CreateTablet for tablet 2d8219e0438e46f783037537e687f945 (DEFAULT_TABLE table=default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:23.824853 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d8219e0438e46f783037537e687f945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:23.829780 22157 tablet_replica.cc:330] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:23.830086 22157 raft_consensus.cc:2229] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:23.830477 22157 raft_consensus.cc:2258] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:23.831161 22157 ts_tablet_manager.cc:1905] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 02:45:23.843091 21436 consensus_peers.cc:487] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 80d6ff40e16d4114a2dc9658f6e63a3b. This is attempt 1: this message will repeat every 5th retry.
I20260806 02:45:23.844008 22153 tablet_replica.cc:330] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:23.844239 22153 raft_consensus.cc:2229] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:45:23.844969 22153 pending_rounds.cc:62] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5: Trying to abort 2 pending ops.
I20260806 02:45:23.845189 22153 pending_rounds.cc:68] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7315391790444490752 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "80d6ff40e16d4114a2dc9658f6e63a3b" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.table_e24bd493305d46a4977ed544ce47cc83" dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" new_extra_config { } }
W20260806 02:45:23.850363 22153 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33104: Aborted: Op aborted
I20260806 02:45:23.851308 22153 pending_rounds.cc:68] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315391788370104320 op_type: NO_OP noop_request { }
I20260806 02:45:23.851753 22153 raft_consensus.cc:2866] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260806 02:45:23.852026 22153 raft_consensus.cc:2258] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:23.859234 22153 ts_tablet_manager.cc:1905] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:23.868587 22157 ts_tablet_manager.cc:1918] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:23.868770 22157 log.cc:1196] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/80d6ff40e16d4114a2dc9658f6e63a3b
I20260806 02:45:23.874836 22153 ts_tablet_manager.cc:1918] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:45:23.875025 22153 log.cc:1196] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/80d6ff40e16d4114a2dc9658f6e63a3b
I20260806 02:45:23.875584 22153 ts_tablet_manager.cc:1939] T 80d6ff40e16d4114a2dc9658f6e63a3b P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:23.876997 22157 ts_tablet_manager.cc:1939] T 80d6ff40e16d4114a2dc9658f6e63a3b P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
W20260806 02:45:23.878283 22091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20260806 02:45:23.878954 22160 tablet_bootstrap.cc:492] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
W20260806 02:45:23.878953 22091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:45:23.879166 22091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 02:45:23.881798 22164 tablet_bootstrap.cc:492] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:23.885145 22164 tablet_bootstrap.cc:654] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:23.886389 21064 catalog_manager.cc:4841] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): tablet 80d6ff40e16d4114a2dc9658f6e63a3b (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]) successfully deleted
W20260806 02:45:23.887007 21064 catalog_manager.cc:4920] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): alter failed for tablet 80d6ff40e16d4114a2dc9658f6e63a3b (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]): Aborted: Op aborted
I20260806 02:45:23.887430 21064 catalog_manager.cc:4841] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): tablet 80d6ff40e16d4114a2dc9658f6e63a3b (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]) successfully deleted
W20260806 02:45:23.894625 22091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:45:23.894786 22091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:45:23.894961 22091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:45:23.895156 22091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:45:23.897166 22160 tablet_bootstrap.cc:654] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:23.907289 22164 tablet_bootstrap.cc:492] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:23.907696 22164 ts_tablet_manager.cc:1397] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.037s user 0.012s sys 0.000s
I20260806 02:45:23.917054 22091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.60.20:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.61:36797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.60.61
--webserver_port=46303
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 01:55:54 UTC on ccycloud.quasar-csdhqu.root.comops.site
build id 4569536
ASAN enabled
I20260806 02:45:23.919476 22091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:45:23.923043 22091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 02:45:23.926468 22164 raft_consensus.cc:348] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:23.928735 22164 raft_consensus.cc:374] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:23.929071 22164 raft_consensus.cc:729] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:23.930187 22164 consensus_queue.cc:260] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:23.938262 22164 ts_tablet_manager.cc:1428] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.030s user 0.003s sys 0.000s
I20260806 02:45:23.951089 22100 consensus_queue.cc:1035] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:23.973179 21080 catalog_manager.cc:3411] renamed table default.table_00f2c397972f4d8e828d61d7821abc00 [id=610932bdd3f94b4e886d586c28ff6249] in HMS: new name default.table_21221d53bf9f4c46a5ffe671a1ffe424
I20260806 02:45:23.975458 22160 tablet_bootstrap.cc:492] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:23.975710 22160 ts_tablet_manager.cc:1397] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.097s user 0.017s sys 0.013s
I20260806 02:45:23.976908 22160 raft_consensus.cc:348] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:23.977241 22160 raft_consensus.cc:374] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:23.977572 22160 raft_consensus.cc:729] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
W20260806 02:45:23.980914 21064 catalog_manager.cc:4914] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): alter failed for tablet 80d6ff40e16d4114a2dc9658f6e63a3b (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]),no further retry: Not found: Tablet not found: 80d6ff40e16d4114a2dc9658f6e63a3b
I20260806 02:45:23.978279 22160 consensus_queue.cc:260] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:23.988226 22160 ts_tablet_manager.cc:1428] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.012s user 0.000s sys 0.004s
W20260806 02:45:24.002156 22180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:24.003193 22178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:45:24.028514 21081 catalog_manager.cc:2148] created HMS catalog entry for table [id=383ba93bcba140c1b1bd2b36f8720b86]
I20260806 02:45:24.038554 21081 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:58094 (request call id 20) took 1501 ms. Trace:
I20260806 02:45:24.052142 22091 server_base.cc:1034] running on GCE node
I20260806 02:45:24.053367 21682 tablet_service.cc:1467] Processing CreateTablet for tablet 237439c720344cfd97bd04bf8368a375 (DEFAULT_TABLE table=default.table_ea1a38afd92a4b15bfabb80e58f8c644 [id=383ba93bcba140c1b1bd2b36f8720b86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:24.053882 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 237439c720344cfd97bd04bf8368a375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:24.053917 21494 tablet_service.cc:1467] Processing CreateTablet for tablet 237439c720344cfd97bd04bf8368a375 (DEFAULT_TABLE table=default.table_ea1a38afd92a4b15bfabb80e58f8c644 [id=383ba93bcba140c1b1bd2b36f8720b86]), partition=RANGE (key) PARTITION UNBOUNDED
W20260806 02:45:24.053996 22183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:45:24.054363 21494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 237439c720344cfd97bd04bf8368a375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:24.055862 22091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:45:24.057869 21863 tablet_service.cc:1467] Processing CreateTablet for tablet 237439c720344cfd97bd04bf8368a375 (DEFAULT_TABLE table=default.table_ea1a38afd92a4b15bfabb80e58f8c644 [id=383ba93bcba140c1b1bd2b36f8720b86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:24.058444 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 237439c720344cfd97bd04bf8368a375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:24.038812 21081 rpcz_store.cc:276] 0806 02:45:22.537225 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:22.619421 (+ 82196us) service_pool.cc:227] Handling call
0806 02:45:23.531511 (+912090us) catalog_manager.cc:2075] Acquired catalog manager lock
0806 02:45:23.532159 (+ 648us) catalog_manager.cc:2121] Created new table and tablet info
0806 02:45:24.028348 (+496189us) catalog_manager.cc:2147] Created new table in HMS catalog
0806 02:45:24.037988 (+ 9640us) catalog_manager.cc:2181] Wrote table and tablets to system table
0806 02:45:24.038196 (+ 208us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0806 02:45:24.038349 (+ 153us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0806 02:45:24.029781 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0806 02:45:24.030029 (+ 248us) write_op.cc:432] Acquiring schema lock in shared mode
0806 02:45:24.030041 (+ 12us) write_op.cc:435] Acquired schema lock
0806 02:45:24.030045 (+ 4us) tablet.cc:660] Decoding operations
0806 02:45:24.030205 (+ 160us) write_op.cc:620] Acquiring the partition lock for write op
0806 02:45:24.030239 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0806 02:45:24.030248 (+ 9us) tablet.cc:683] Acquiring locks for 2 operations
0806 02:45:24.030376 (+ 128us) tablet.cc:699] Row locks acquired
0806 02:45:24.030381 (+ 5us) write_op.cc:260] PREPARE: finished
0806 02:45:24.030427 (+ 46us) write_op.cc:270] Start()
0806 02:45:24.030477 (+ 50us) write_op.cc:276] Timestamp: P: 1785984324030084 usec, L: 0
0806 02:45:24.030483 (+ 6us) op_driver.cc:331] REPLICATION: starting
0806 02:45:24.030758 (+ 275us) log.cc:842] Serialized 432 byte log entry
0806 02:45:24.035831 (+ 5073us) op_driver.cc:444] REPLICATION: finished
0806 02:45:24.035998 (+ 167us) write_op.cc:301] APPLY: starting
0806 02:45:24.036021 (+ 23us) tablet.cc:1365] starting BulkCheckPresence
0806 02:45:24.036100 (+ 79us) tablet.cc:1368] finished BulkCheckPresence
0806 02:45:24.036198 (+ 98us) tablet.cc:1370] starting ApplyRowOperation cycle
0806 02:45:24.036441 (+ 243us) tablet.cc:1381] finished ApplyRowOperation cycle
0806 02:45:24.036525 (+ 84us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0806 02:45:24.036536 (+ 11us) write_op.cc:312] APPLY: finished
0806 02:45:24.036649 (+ 113us) log.cc:842] Serialized 28 byte log entry
0806 02:45:24.036774 (+ 125us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:24.036805 (+ 31us) write_op.cc:454] Released schema lock
0806 02:45:24.036815 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0806 02:45:24.036916 (+ 101us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:24.036923 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":440504,"Hive Metastore.run_cpu_time_us":405,"Hive Metastore.run_wall_time_us":55049,"child_traces":[["op",{"apply.queue_time_us":103,"apply.run_cpu_time_us":1020,"apply.run_wall_time_us":1021,"num_ops":2,"prepare.queue_time_us":193,"prepare.run_cpu_time_us":1137,"prepare.run_wall_time_us":1138,"raft.queue_time_us":2007,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":350,"replication_time_us":5314}]]}
I20260806 02:45:24.062433 22150 tablet_bootstrap.cc:492] T 237439c720344cfd97bd04bf8368a375 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:24.063721 21081 catalog_manager.cc:5587] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655) does not have the latest schema for tablet 2d8219e0438e46f783037537e687f945 (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]). Expected version 1 got 0
I20260806 02:45:24.063918 22091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:45:24.065498 22150 tablet_bootstrap.cc:654] T 237439c720344cfd97bd04bf8368a375 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
W20260806 02:45:24.067615 21081 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 2d8219e0438e46f783037537e687f945 (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 2d8219e0438e46f783037537e687f945 consensus state has no leader
I20260806 02:45:24.070788 22160 tablet_bootstrap.cc:492] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:24.073477 22160 tablet_bootstrap.cc:654] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:24.084551 22160 tablet_bootstrap.cc:492] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:24.084823 22160 ts_tablet_manager.cc:1397] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.007s
I20260806 02:45:24.085644 22164 tablet_bootstrap.cc:492] T 237439c720344cfd97bd04bf8368a375 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:24.085824 22160 raft_consensus.cc:348] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.086190 22160 raft_consensus.cc:374] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:24.086351 22160 raft_consensus.cc:729] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:24.086854 22160 consensus_queue.cc:260] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.088376 22164 tablet_bootstrap.cc:654] T 237439c720344cfd97bd04bf8368a375 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:24.089154 22160 ts_tablet_manager.cc:1428] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 02:45:24.091589 22150 tablet_bootstrap.cc:492] T 237439c720344cfd97bd04bf8368a375 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:24.091809 22150 ts_tablet_manager.cc:1397] T 237439c720344cfd97bd04bf8368a375 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.030s user 0.013s sys 0.011s
I20260806 02:45:24.092721 22150 raft_consensus.cc:348] T 237439c720344cfd97bd04bf8368a375 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.093389 22150 raft_consensus.cc:374] T 237439c720344cfd97bd04bf8368a375 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:24.093500 22150 raft_consensus.cc:729] T 237439c720344cfd97bd04bf8368a375 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:24.093827 22150 consensus_queue.cc:260] T 237439c720344cfd97bd04bf8368a375 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.095330 22150 ts_tablet_manager.cc:1428] T 237439c720344cfd97bd04bf8368a375 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 02:45:24.101116 21081 catalog_manager.cc:5587] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593) does not have the latest schema for tablet 2d8219e0438e46f783037537e687f945 (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]). Expected version 1 got 0
I20260806 02:45:24.101549 22164 tablet_bootstrap.cc:492] T 237439c720344cfd97bd04bf8368a375 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:24.101743 22164 ts_tablet_manager.cc:1397] T 237439c720344cfd97bd04bf8368a375 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.005s
I20260806 02:45:24.101888 22091 hybrid_clock.cc:648] HybridClock initialized: now 1785984324085823 us; error 15970 us; skew 500 ppm
I20260806 02:45:24.102579 22091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 02:45:24.103047 22164 raft_consensus.cc:348] T 237439c720344cfd97bd04bf8368a375 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.103294 22164 raft_consensus.cc:374] T 237439c720344cfd97bd04bf8368a375 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:24.103391 22164 raft_consensus.cc:729] T 237439c720344cfd97bd04bf8368a375 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:24.103649 22164 consensus_queue.cc:260] T 237439c720344cfd97bd04bf8368a375 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.103615 22121 consensus_queue.cc:1035] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:24.106416 21081 catalog_manager.cc:5587] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883) does not have the latest schema for tablet 2d8219e0438e46f783037537e687f945 (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]). Expected version 1 got 0
I20260806 02:45:24.107620 22164 ts_tablet_manager.cc:1428] T 237439c720344cfd97bd04bf8368a375 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260806 02:45:24.131148 21074 catalog_manager.cc:2148] created HMS catalog entry for table [id=c6a05485b0a54967ba8904aa969edd3f]
I20260806 02:45:24.141731 21081 master_service.cc:946] ReplaceTablet: received request to replace tablet d0956f9352c84a069760fa70da9e8881 from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:24.142616 22052 consensus_queue.cc:1035] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:24.144352 21074 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:58094 (request call id 23) took 1481 ms. Trace:
I20260806 02:45:24.147941 22196 raft_consensus.cc:482] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:24.148200 22196 raft_consensus.cc:504] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.155364 22121 consensus_queue.cc:1035] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:24.162528 21707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "237439c720344cfd97bd04bf8368a375" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" is_pre_election: true
I20260806 02:45:24.162914 21707 raft_consensus.cc:2454] T 237439c720344cfd97bd04bf8368a375 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 0.
I20260806 02:45:24.163339 21863 tablet_service.cc:1467] Processing CreateTablet for tablet 3196f6ee70d34d298b7db56d9759fab7 (DEFAULT_TABLE table=default.table_00c06d2b7d6d40658a4390f4ee3bbab5 [id=c6a05485b0a54967ba8904aa969edd3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:24.163928 21436 leader_election.cc:302] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f; no voters:
I20260806 02:45:24.164145 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3196f6ee70d34d298b7db56d9759fab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:24.164706 22121 raft_consensus.cc:2781] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:24.165022 22121 raft_consensus.cc:482] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:24.165221 22121 raft_consensus.cc:3037] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:24.165674 22196 leader_election.cc:288] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:24.151386 22195 consensus_queue.cc:1035] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260806 02:45:24.166440 21682 tablet_service.cc:1467] Processing CreateTablet for tablet 3196f6ee70d34d298b7db56d9759fab7 (DEFAULT_TABLE table=default.table_00c06d2b7d6d40658a4390f4ee3bbab5 [id=c6a05485b0a54967ba8904aa969edd3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:24.167085 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3196f6ee70d34d298b7db56d9759fab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:24.144605 21074 rpcz_store.cc:276] 0806 02:45:22.662598 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:22.909877 (+247279us) service_pool.cc:227] Handling call
0806 02:45:23.531695 (+621818us) catalog_manager.cc:2075] Acquired catalog manager lock
0806 02:45:23.532309 (+ 614us) catalog_manager.cc:2121] Created new table and tablet info
0806 02:45:24.131097 (+598788us) catalog_manager.cc:2147] Created new table in HMS catalog
0806 02:45:24.144064 (+ 12967us) catalog_manager.cc:2181] Wrote table and tablets to system table
0806 02:45:24.144175 (+ 111us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0806 02:45:24.144285 (+ 110us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0806 02:45:24.132586 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0806 02:45:24.132831 (+ 245us) write_op.cc:432] Acquiring schema lock in shared mode
0806 02:45:24.132842 (+ 11us) write_op.cc:435] Acquired schema lock
0806 02:45:24.132846 (+ 4us) tablet.cc:660] Decoding operations
0806 02:45:24.132989 (+ 143us) write_op.cc:620] Acquiring the partition lock for write op
0806 02:45:24.133011 (+ 22us) write_op.cc:641] Partition lock acquired for write op
0806 02:45:24.133031 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0806 02:45:24.133385 (+ 354us) tablet.cc:699] Row locks acquired
0806 02:45:24.133391 (+ 6us) write_op.cc:260] PREPARE: finished
0806 02:45:24.133452 (+ 61us) write_op.cc:270] Start()
0806 02:45:24.133506 (+ 54us) write_op.cc:276] Timestamp: P: 1785984324133096 usec, L: 0
0806 02:45:24.133512 (+ 6us) op_driver.cc:331] REPLICATION: starting
0806 02:45:24.133774 (+ 262us) log.cc:842] Serialized 432 byte log entry
0806 02:45:24.139832 (+ 6058us) op_driver.cc:444] REPLICATION: finished
0806 02:45:24.140572 (+ 740us) write_op.cc:301] APPLY: starting
0806 02:45:24.140592 (+ 20us) tablet.cc:1365] starting BulkCheckPresence
0806 02:45:24.140644 (+ 52us) tablet.cc:1368] finished BulkCheckPresence
0806 02:45:24.140687 (+ 43us) tablet.cc:1370] starting ApplyRowOperation cycle
0806 02:45:24.140804 (+ 117us) tablet.cc:1381] finished ApplyRowOperation cycle
0806 02:45:24.140843 (+ 39us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0806 02:45:24.140850 (+ 7us) write_op.cc:312] APPLY: finished
0806 02:45:24.140937 (+ 87us) log.cc:842] Serialized 28 byte log entry
0806 02:45:24.141008 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:24.141042 (+ 34us) write_op.cc:454] Released schema lock
0806 02:45:24.141049 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0806 02:45:24.141107 (+ 58us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:24.141111 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":495505,"Hive Metastore.run_cpu_time_us":423,"Hive Metastore.run_wall_time_us":102998,"child_traces":[["op",{"apply.queue_time_us":662,"apply.run_cpu_time_us":619,"apply.run_wall_time_us":618,"num_ops":2,"prepare.queue_time_us":303,"prepare.run_cpu_time_us":1319,"prepare.run_wall_time_us":1320,"raft.queue_time_us":2242,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":335,"replication_time_us":6287}]]}
I20260806 02:45:24.170140 21074 master_service.cc:946] ReplaceTablet: received request to replace tablet fde40ab726344932997271f125940c61 from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:24.183578 21081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Sending DeleteTablet for 3 replicas of tablet d0956f9352c84a069760fa70da9e8881
I20260806 02:45:24.191581 22121 raft_consensus.cc:504] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.186869 21074 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Sending DeleteTablet for 3 replicas of tablet fde40ab726344932997271f125940c61
I20260806 02:45:24.192570 22121 leader_election.cc:288] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 election: Requested vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:24.192684 22100 consensus_queue.cc:1035] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:24.193881 21865 tablet_service.cc:1514] Processing DeleteTablet for tablet fde40ab726344932997271f125940c61 with delete_type TABLET_DATA_DELETED (replaced by tablet b4e3f2c8eb2c4ea8a1b041be073bdd08) from {username='slave'} at 127.0.0.1:56372
I20260806 02:45:24.194486 22157 tablet_replica.cc:330] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:24.194525 21074 catalog_manager.cc:6565] ReplaceTablet: tablet fde40ab726344932997271f125940c61 deleted and replaced by tablet b4e3f2c8eb2c4ea8a1b041be073bdd08
I20260806 02:45:24.194682 21074 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:58094 (request call id 29) took 1402 ms. Trace:
I20260806 02:45:24.194945 22157 raft_consensus.cc:2229] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:24.195242 22157 raft_consensus.cc:2258] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:24.194924 21074 rpcz_store.cc:276] 0806 02:45:22.792715 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:24.169928 (+1377213us) service_pool.cc:227] Handling call
0806 02:45:24.194638 (+ 24710us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0806 02:45:24.172354 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0806 02:45:24.172507 (+ 153us) write_op.cc:432] Acquiring schema lock in shared mode
0806 02:45:24.172518 (+ 11us) write_op.cc:435] Acquired schema lock
0806 02:45:24.172522 (+ 4us) tablet.cc:660] Decoding operations
0806 02:45:24.172692 (+ 170us) write_op.cc:620] Acquiring the partition lock for write op
0806 02:45:24.172713 (+ 21us) write_op.cc:641] Partition lock acquired for write op
0806 02:45:24.172721 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0806 02:45:24.172862 (+ 141us) tablet.cc:699] Row locks acquired
0806 02:45:24.172866 (+ 4us) write_op.cc:260] PREPARE: finished
0806 02:45:24.172958 (+ 92us) write_op.cc:270] Start()
0806 02:45:24.173007 (+ 49us) write_op.cc:276] Timestamp: P: 1785984324172559 usec, L: 0
0806 02:45:24.173013 (+ 6us) op_driver.cc:331] REPLICATION: starting
0806 02:45:24.173283 (+ 270us) log.cc:842] Serialized 655 byte log entry
0806 02:45:24.184993 (+ 11710us) op_driver.cc:444] REPLICATION: finished
0806 02:45:24.185381 (+ 388us) write_op.cc:301] APPLY: starting
0806 02:45:24.185406 (+ 25us) tablet.cc:1365] starting BulkCheckPresence
0806 02:45:24.185467 (+ 61us) tablet.cc:1368] finished BulkCheckPresence
0806 02:45:24.185600 (+ 133us) tablet.cc:1370] starting ApplyRowOperation cycle
0806 02:45:24.185913 (+ 313us) tablet.cc:1381] finished ApplyRowOperation cycle
0806 02:45:24.186000 (+ 87us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0806 02:45:24.186011 (+ 11us) write_op.cc:312] APPLY: finished
0806 02:45:24.186119 (+ 108us) log.cc:842] Serialized 28 byte log entry
0806 02:45:24.186246 (+ 127us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:24.186274 (+ 28us) write_op.cc:454] Released schema lock
0806 02:45:24.186281 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0806 02:45:24.186411 (+ 130us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:24.186415 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":272,"apply.run_cpu_time_us":1152,"apply.run_wall_time_us":1152,"num_ops":2,"prepare.queue_time_us":269,"prepare.run_cpu_time_us":1085,"prepare.run_wall_time_us":1085,"raft.queue_time_us":861,"raft.run_cpu_time_us":9,"raft.run_wall_time_us":10,"replication_time_us":11942}]]}
I20260806 02:45:24.196165 21705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "237439c720344cfd97bd04bf8368a375" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
I20260806 02:45:24.196530 21705 raft_consensus.cc:3037] T 237439c720344cfd97bd04bf8368a375 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:24.198721 21865 tablet_service.cc:1514] Processing DeleteTablet for tablet d0956f9352c84a069760fa70da9e8881 with delete_type TABLET_DATA_DELETED (replaced by tablet 3d44c58ba1f044b38f10da10cec410e3) from {username='slave'} at 127.0.0.1:56372
I20260806 02:45:24.199363 21081 catalog_manager.cc:6565] ReplaceTablet: tablet d0956f9352c84a069760fa70da9e8881 deleted and replaced by tablet 3d44c58ba1f044b38f10da10cec410e3
I20260806 02:45:24.199576 21081 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:58094 (request call id 28) took 1414 ms. Trace:
I20260806 02:45:24.199714 21081 rpcz_store.cc:276] 0806 02:45:22.785201 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:24.141444 (+1356243us) service_pool.cc:227] Handling call
0806 02:45:24.199534 (+ 58090us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0806 02:45:24.147198 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0806 02:45:24.147325 (+ 127us) write_op.cc:432] Acquiring schema lock in shared mode
0806 02:45:24.147332 (+ 7us) write_op.cc:435] Acquired schema lock
0806 02:45:24.147335 (+ 3us) tablet.cc:660] Decoding operations
0806 02:45:24.147479 (+ 144us) write_op.cc:620] Acquiring the partition lock for write op
0806 02:45:24.147494 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0806 02:45:24.147498 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
0806 02:45:24.147587 (+ 89us) tablet.cc:699] Row locks acquired
0806 02:45:24.147590 (+ 3us) write_op.cc:260] PREPARE: finished
0806 02:45:24.147631 (+ 41us) write_op.cc:270] Start()
0806 02:45:24.147664 (+ 33us) write_op.cc:276] Timestamp: P: 1785984324147278 usec, L: 0
0806 02:45:24.147668 (+ 4us) op_driver.cc:331] REPLICATION: starting
0806 02:45:24.147873 (+ 205us) log.cc:842] Serialized 655 byte log entry
0806 02:45:24.156306 (+ 8433us) op_driver.cc:444] REPLICATION: finished
0806 02:45:24.156636 (+ 330us) write_op.cc:301] APPLY: starting
0806 02:45:24.156654 (+ 18us) tablet.cc:1365] starting BulkCheckPresence
0806 02:45:24.156705 (+ 51us) tablet.cc:1368] finished BulkCheckPresence
0806 02:45:24.156740 (+ 35us) tablet.cc:1370] starting ApplyRowOperation cycle
0806 02:45:24.156851 (+ 111us) tablet.cc:1381] finished ApplyRowOperation cycle
0806 02:45:24.156929 (+ 78us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0806 02:45:24.156936 (+ 7us) write_op.cc:312] APPLY: finished
0806 02:45:24.157026 (+ 90us) log.cc:842] Serialized 28 byte log entry
0806 02:45:24.157115 (+ 89us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:24.157146 (+ 31us) write_op.cc:454] Released schema lock
0806 02:45:24.157154 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0806 02:45:24.157232 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:24.157236 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":270,"apply.run_cpu_time_us":672,"apply.run_wall_time_us":661,"num_ops":2,"prepare.queue_time_us":1534,"prepare.run_cpu_time_us":790,"prepare.run_wall_time_us":790,"raft.queue_time_us":1326,"raft.run_cpu_time_us":8,"raft.run_wall_time_us":9,"replication_time_us":8611}]]}
I20260806 02:45:24.201889 21705 raft_consensus.cc:2454] T 237439c720344cfd97bd04bf8368a375 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 1.
I20260806 02:45:24.203876 22157 ts_tablet_manager.cc:1905] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:24.206048 22164 tablet_bootstrap.cc:492] T 3196f6ee70d34d298b7db56d9759fab7 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:24.207166 21081 master_service.cc:946] ReplaceTablet: received request to replace tablet cba792f10b234bc9aa7b3189fbbf1ea4 from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:24.210084 21436 leader_election.cc:302] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f; no voters:
I20260806 02:45:24.210541 22121 raft_consensus.cc:2781] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:24.210716 22121 raft_consensus.cc:686] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 LEADER]: Becoming Leader. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Running, Role: LEADER
I20260806 02:45:24.211118 22121 consensus_queue.cc:237] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.216187 21683 tablet_service.cc:1514] Processing DeleteTablet for tablet fde40ab726344932997271f125940c61 with delete_type TABLET_DATA_DELETED (replaced by tablet b4e3f2c8eb2c4ea8a1b041be073bdd08) from {username='slave'} at 127.0.0.1:38404
I20260806 02:45:24.216626 22149 tablet_replica.cc:330] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:24.217137 21074 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58094:
table { table_name: "default.Table_a7715d8e001f48b2899c3c2a56a18a06" } modify_external_catalogs: true
I20260806 02:45:24.217515 21074 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58094:
table { table_name: "default.Table_a7715d8e001f48b2899c3c2a56a18a06" } modify_external_catalogs: true
I20260806 02:45:24.217732 21683 tablet_service.cc:1514] Processing DeleteTablet for tablet d0956f9352c84a069760fa70da9e8881 with delete_type TABLET_DATA_DELETED (replaced by tablet 3d44c58ba1f044b38f10da10cec410e3) from {username='slave'} at 127.0.0.1:38404
I20260806 02:45:24.224040 22091 webserver.cc:460] Webserver started at http://127.20.60.61:46303/ using document root <none> and password file <none>
I20260806 02:45:24.228412 22149 raft_consensus.cc:2229] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:24.229110 22149 pending_rounds.cc:62] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f: Trying to abort 1 pending ops.
I20260806 02:45:24.229475 22149 pending_rounds.cc:68] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315391787999129600 op_type: NO_OP noop_request { }
I20260806 02:45:24.229871 22149 raft_consensus.cc:2866] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260806 02:45:24.230141 22149 raft_consensus.cc:2258] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:24.230451 22091 fs_manager.cc:359] Metadata directory not provided
I20260806 02:45:24.230762 22091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:45:24.231731 22164 tablet_bootstrap.cc:654] T 3196f6ee70d34d298b7db56d9759fab7 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
W20260806 02:45:24.231206 22206 consensus_queue.cc:1159] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260806 02:45:24.233704 22149 ts_tablet_manager.cc:1905] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 02:45:24.239259 21436 consensus_peers.cc:487] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260806 02:45:24.242898 22100 consensus_queue.cc:1035] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:24.244112 22149 ts_tablet_manager.cc:1918] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:24.244369 22149 log.cc:1196] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/fde40ab726344932997271f125940c61
I20260806 02:45:24.245388 22149 ts_tablet_manager.cc:1939] T fde40ab726344932997271f125940c61 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:24.245756 22052 consensus_queue.cc:1035] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:24.247294 22149 tablet_replica.cc:330] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:24.248914 22149 raft_consensus.cc:2229] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:45:24.249858 22149 raft_consensus.cc:2258] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:24.251515 22149 ts_tablet_manager.cc:1905] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:24.264679 22157 ts_tablet_manager.cc:1918] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:24.264986 22157 log.cc:1196] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/fde40ab726344932997271f125940c61
I20260806 02:45:24.265749 22157 ts_tablet_manager.cc:1939] T fde40ab726344932997271f125940c61 P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:24.267247 21064 catalog_manager.cc:4841] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): tablet fde40ab726344932997271f125940c61 (table default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443]) successfully deleted
I20260806 02:45:24.267894 22214 consensus_queue.cc:1035] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 02:45:24.275477 21081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Sending DeleteTablet for 3 replicas of tablet cba792f10b234bc9aa7b3189fbbf1ea4
I20260806 02:45:24.276103 21081 catalog_manager.cc:6565] ReplaceTablet: tablet cba792f10b234bc9aa7b3189fbbf1ea4 deleted and replaced by tablet d571f1c27e944cb4874a6c30a141c2ad
I20260806 02:45:24.276425 21081 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:58094 (request call id 31) took 1433 ms. Trace:
I20260806 02:45:24.276640 21081 rpcz_store.cc:276] 0806 02:45:22.843294 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:24.207017 (+1363723us) service_pool.cc:227] Handling call
0806 02:45:24.217884 (+ 10867us) spinlock_profiling.cc:243] Waited 6.39 ms on lock 0x615000033024. stack: 00007f1a1efb666e 00007f1a1efb6430 00007f1a1e03d5ed 00000000008ea064 00000000008ea02d 00007f1a2d6cf27d 00007f1a2d6cf373 00007f1a2a4f1361 00007f1a31e6466a 00007f1a31ff8527 00007f1a292eada7 00007f1a292eaad5 00007f1a25af29be 00007f1a25af179e 00007f1a25af7b25 00007f1a25af854f
0806 02:45:24.276213 (+ 58329us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0806 02:45:24.219317 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0806 02:45:24.219441 (+ 124us) write_op.cc:432] Acquiring schema lock in shared mode
0806 02:45:24.219449 (+ 8us) write_op.cc:435] Acquired schema lock
0806 02:45:24.219453 (+ 4us) tablet.cc:660] Decoding operations
0806 02:45:24.219647 (+ 194us) write_op.cc:620] Acquiring the partition lock for write op
0806 02:45:24.219660 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0806 02:45:24.219668 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0806 02:45:24.219777 (+ 109us) tablet.cc:699] Row locks acquired
0806 02:45:24.219782 (+ 5us) write_op.cc:260] PREPARE: finished
0806 02:45:24.219820 (+ 38us) write_op.cc:270] Start()
0806 02:45:24.219856 (+ 36us) write_op.cc:276] Timestamp: P: 1785984324219461 usec, L: 0
0806 02:45:24.219862 (+ 6us) op_driver.cc:331] REPLICATION: starting
0806 02:45:24.220056 (+ 194us) log.cc:842] Serialized 655 byte log entry
0806 02:45:24.268465 (+ 48409us) op_driver.cc:444] REPLICATION: finished
0806 02:45:24.269374 (+ 909us) write_op.cc:301] APPLY: starting
0806 02:45:24.269389 (+ 15us) tablet.cc:1365] starting BulkCheckPresence
0806 02:45:24.269489 (+ 100us) tablet.cc:1368] finished BulkCheckPresence
0806 02:45:24.269521 (+ 32us) tablet.cc:1370] starting ApplyRowOperation cycle
0806 02:45:24.269645 (+ 124us) tablet.cc:1381] finished ApplyRowOperation cycle
0806 02:45:24.269698 (+ 53us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0806 02:45:24.269703 (+ 5us) write_op.cc:312] APPLY: finished
0806 02:45:24.269764 (+ 61us) log.cc:842] Serialized 28 byte log entry
0806 02:45:24.269830 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:24.269852 (+ 22us) write_op.cc:454] Released schema lock
0806 02:45:24.269860 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0806 02:45:24.269921 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:24.269925 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":14686464,"child_traces":[["op",{"apply.queue_time_us":837,"apply.run_cpu_time_us":692,"apply.run_wall_time_us":690,"num_ops":2,"prepare.queue_time_us":314,"raft.queue_time_us":3147,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":8,"replication_time_us":48581,"threads_started":1}]]}
I20260806 02:45:24.273842 21863 tablet_service.cc:1467] Processing CreateTablet for tablet b4e3f2c8eb2c4ea8a1b041be073bdd08 (DEFAULT_TABLE table=default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:24.274734 21865 tablet_service.cc:1467] Processing CreateTablet for tablet 3d44c58ba1f044b38f10da10cec410e3 (DEFAULT_TABLE table=default.table_910edc3d61454438a69af3c32d9b721f [id=1bbd1184e7754c5bb9ffdf82791c73fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:24.278463 21862 tablet_service.cc:1514] Processing DeleteTablet for tablet cba792f10b234bc9aa7b3189fbbf1ea4 with delete_type TABLET_DATA_DELETED (replaced by tablet d571f1c27e944cb4874a6c30a141c2ad) from {username='slave'} at 127.0.0.1:56372
I20260806 02:45:24.281308 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4e3f2c8eb2c4ea8a1b041be073bdd08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:24.285743 21865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d44c58ba1f044b38f10da10cec410e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:24.302006 22159 consensus_queue.cc:1035] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260806 02:45:24.307516 21434 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.60.3:43883, user_credentials={real_user=slave}} blocked reactor thread for 188607us
I20260806 02:45:24.313485 22159 consensus_queue.cc:1035] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:24.315505 21887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "237439c720344cfd97bd04bf8368a375" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:24.315855 21887 raft_consensus.cc:2454] T 237439c720344cfd97bd04bf8368a375 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 0.
I20260806 02:45:24.316726 21887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "237439c720344cfd97bd04bf8368a375" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:24.317121 21887 raft_consensus.cc:3037] T 237439c720344cfd97bd04bf8368a375 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:24.322068 21887 raft_consensus.cc:2454] T 237439c720344cfd97bd04bf8368a375 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 1.
I20260806 02:45:24.326742 22157 tablet_replica.cc:330] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:24.326972 22150 tablet_bootstrap.cc:492] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:24.327135 22157 raft_consensus.cc:2229] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:24.327581 22157 raft_consensus.cc:2258] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:24.329300 22157 ts_tablet_manager.cc:1905] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:24.329615 22150 tablet_bootstrap.cc:654] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:24.338828 21494 tablet_service.cc:1467] Processing CreateTablet for tablet 3196f6ee70d34d298b7db56d9759fab7 (DEFAULT_TABLE table=default.table_00c06d2b7d6d40658a4390f4ee3bbab5 [id=c6a05485b0a54967ba8904aa969edd3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:24.339349 21494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3196f6ee70d34d298b7db56d9759fab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:24.350906 22160 tablet_bootstrap.cc:492] T 3196f6ee70d34d298b7db56d9759fab7 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:24.354022 22160 tablet_bootstrap.cc:654] T 3196f6ee70d34d298b7db56d9759fab7 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:24.356626 22198 consensus_queue.cc:1035] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 02:45:24.368180 21495 tablet_service.cc:1514] Processing DeleteTablet for tablet d0956f9352c84a069760fa70da9e8881 with delete_type TABLET_DATA_DELETED (replaced by tablet 3d44c58ba1f044b38f10da10cec410e3) from {username='slave'} at 127.0.0.1:33104
I20260806 02:45:24.368692 22153 tablet_replica.cc:330] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:24.368995 22153 raft_consensus.cc:2229] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:24.369295 22153 raft_consensus.cc:2258] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:24.370469 21495 tablet_service.cc:1467] Processing CreateTablet for tablet b4e3f2c8eb2c4ea8a1b041be073bdd08 (DEFAULT_TABLE table=default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:24.370553 22153 ts_tablet_manager.cc:1905] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:24.370962 21495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4e3f2c8eb2c4ea8a1b041be073bdd08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:24.378504 21494 tablet_service.cc:1514] Processing DeleteTablet for tablet fde40ab726344932997271f125940c61 with delete_type TABLET_DATA_DELETED (replaced by tablet b4e3f2c8eb2c4ea8a1b041be073bdd08) from {username='slave'} at 127.0.0.1:33104
I20260806 02:45:24.384197 22153 ts_tablet_manager.cc:1918] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:24.384413 22153 log.cc:1196] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/d0956f9352c84a069760fa70da9e8881
I20260806 02:45:24.384963 22153 ts_tablet_manager.cc:1939] T d0956f9352c84a069760fa70da9e8881 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:24.386380 22153 tablet_replica.cc:330] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:24.386667 22153 raft_consensus.cc:2229] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:45:24.387137 22153 raft_consensus.cc:2258] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:24.397384 21081 master_service.cc:946] ReplaceTablet: received request to replace tablet fde40ab726344932997271f125940c61 from {username='slave'} at 127.0.0.1:58094
W20260806 02:45:24.397612 21617 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.60.1:36593, user_credentials={real_user=slave}} blocked reactor thread for 153199us
I20260806 02:45:24.399749 22157 ts_tablet_manager.cc:1918] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:24.400084 22157 log.cc:1196] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/d0956f9352c84a069760fa70da9e8881
I20260806 02:45:24.400627 21683 tablet_service.cc:1467] Processing CreateTablet for tablet 3d44c58ba1f044b38f10da10cec410e3 (DEFAULT_TABLE table=default.table_910edc3d61454438a69af3c32d9b721f [id=1bbd1184e7754c5bb9ffdf82791c73fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:24.401243 21683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d44c58ba1f044b38f10da10cec410e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:24.403158 22157 ts_tablet_manager.cc:1939] T d0956f9352c84a069760fa70da9e8881 P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:24.404448 22157 tablet_replica.cc:330] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:24.407349 21064 catalog_manager.cc:4841] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): tablet d0956f9352c84a069760fa70da9e8881 (table default.table_910edc3d61454438a69af3c32d9b721f [id=1bbd1184e7754c5bb9ffdf82791c73fb]) successfully deleted
I20260806 02:45:24.410243 21081 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:58094 (request call id 32) took 1566 ms. Trace:
I20260806 02:45:24.407974 21068 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet fde40ab726344932997271f125940c61 (table default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443]) successfully deleted
I20260806 02:45:24.411690 21682 tablet_service.cc:1467] Processing CreateTablet for tablet b4e3f2c8eb2c4ea8a1b041be073bdd08 (DEFAULT_TABLE table=default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:24.411834 21081 rpcz_store.cc:276] 0806 02:45:22.843422 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:24.397157 (+1553735us) service_pool.cc:227] Handling call
0806 02:45:24.410130 (+ 12973us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260806 02:45:24.413100 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4e3f2c8eb2c4ea8a1b041be073bdd08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:24.424340 21064 catalog_manager.cc:4841] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): tablet d0956f9352c84a069760fa70da9e8881 (table default.table_910edc3d61454438a69af3c32d9b721f [id=1bbd1184e7754c5bb9ffdf82791c73fb]) successfully deleted
I20260806 02:45:24.426868 21494 tablet_service.cc:1514] Processing DeleteTablet for tablet cba792f10b234bc9aa7b3189fbbf1ea4 with delete_type TABLET_DATA_DELETED (replaced by tablet d571f1c27e944cb4874a6c30a141c2ad) from {username='slave'} at 127.0.0.1:33104
I20260806 02:45:24.427763 21495 tablet_service.cc:1467] Processing CreateTablet for tablet 3d44c58ba1f044b38f10da10cec410e3 (DEFAULT_TABLE table=default.table_910edc3d61454438a69af3c32d9b721f [id=1bbd1184e7754c5bb9ffdf82791c73fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:24.428566 21495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d44c58ba1f044b38f10da10cec410e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:24.433316 21081 catalog_manager.cc:5495] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1). New cstate: current_term: 1 leader_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260806 02:45:24.435678 21863 tablet_service.cc:1467] Processing CreateTablet for tablet d571f1c27e944cb4874a6c30a141c2ad (DEFAULT_TABLE table=default.table_c94f1c901de74f71bc21408cd63fa3ea [id=2981a18bee6a4705ae5547688e2728f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:24.436264 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d571f1c27e944cb4874a6c30a141c2ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:24.437495 22164 tablet_bootstrap.cc:492] T 3196f6ee70d34d298b7db56d9759fab7 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:24.437734 22164 ts_tablet_manager.cc:1397] T 3196f6ee70d34d298b7db56d9759fab7 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.232s user 0.006s sys 0.006s
I20260806 02:45:24.442803 22091 fs_manager.cc:709] Time spent opening directory manager: real 0.205s user 0.004s sys 0.002s
I20260806 02:45:24.454502 22121 consensus_queue.cc:1035] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:24.456017 22150 tablet_bootstrap.cc:492] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:24.456282 22150 ts_tablet_manager.cc:1397] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.129s user 0.004s sys 0.007s
I20260806 02:45:24.457877 22160 tablet_bootstrap.cc:492] T 3196f6ee70d34d298b7db56d9759fab7 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:24.458235 22160 ts_tablet_manager.cc:1397] T 3196f6ee70d34d298b7db56d9759fab7 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.107s user 0.003s sys 0.009s
I20260806 02:45:24.459017 21683 tablet_service.cc:1514] Processing DeleteTablet for tablet cba792f10b234bc9aa7b3189fbbf1ea4 with delete_type TABLET_DATA_DELETED (replaced by tablet d571f1c27e944cb4874a6c30a141c2ad) from {username='slave'} at 127.0.0.1:38404
I20260806 02:45:24.459230 21681 tablet_service.cc:1467] Processing CreateTablet for tablet d571f1c27e944cb4874a6c30a141c2ad (DEFAULT_TABLE table=default.table_c94f1c901de74f71bc21408cd63fa3ea [id=2981a18bee6a4705ae5547688e2728f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:24.459726 21681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d571f1c27e944cb4874a6c30a141c2ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:24.459517 22160 raft_consensus.cc:348] T 3196f6ee70d34d298b7db56d9759fab7 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.459916 22160 raft_consensus.cc:374] T 3196f6ee70d34d298b7db56d9759fab7 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:24.460114 22160 raft_consensus.cc:729] T 3196f6ee70d34d298b7db56d9759fab7 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:24.460104 22164 raft_consensus.cc:348] T 3196f6ee70d34d298b7db56d9759fab7 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.460558 22164 raft_consensus.cc:374] T 3196f6ee70d34d298b7db56d9759fab7 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:24.460798 22164 raft_consensus.cc:729] T 3196f6ee70d34d298b7db56d9759fab7 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:24.472112 22235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:24.474596 22091 fs_manager.cc:725] Time spent opening block manager: real 0.028s user 0.002s sys 0.005s
I20260806 02:45:24.474908 22091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data,/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22"
format_stamp: "Formatted at 2026-08-06 02:45:17 on dist-test-slave-bw88"
I20260806 02:45:24.477329 22091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:45:24.488206 22159 consensus_queue.cc:1035] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:24.496747 22160 consensus_queue.cc:260] T 3196f6ee70d34d298b7db56d9759fab7 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.457317 22150 raft_consensus.cc:348] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.497605 22150 raft_consensus.cc:374] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:24.497931 22150 raft_consensus.cc:729] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:24.498428 22150 consensus_queue.cc:260] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.499202 22150 ts_tablet_manager.cc:1428] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.043s user 0.000s sys 0.004s
I20260806 02:45:24.499641 22150 tablet_bootstrap.cc:492] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:24.502422 22164 consensus_queue.cc:260] T 3196f6ee70d34d298b7db56d9759fab7 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.503214 22164 ts_tablet_manager.cc:1428] T 3196f6ee70d34d298b7db56d9759fab7 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.065s user 0.003s sys 0.000s
I20260806 02:45:24.503522 22160 ts_tablet_manager.cc:1428] T 3196f6ee70d34d298b7db56d9759fab7 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.045s user 0.003s sys 0.000s
I20260806 02:45:24.503679 22164 tablet_bootstrap.cc:492] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:24.504022 22160 tablet_bootstrap.cc:492] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:24.511638 22157 raft_consensus.cc:2229] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:45:24.512598 22153 ts_tablet_manager.cc:1905] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:24.513290 22157 raft_consensus.cc:2258] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:24.514582 22157 ts_tablet_manager.cc:1905] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:24.528656 22239 raft_consensus.cc:482] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:24.529101 22239 raft_consensus.cc:504] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.530365 22157 ts_tablet_manager.cc:1918] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:24.530629 22157 log.cc:1196] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/cba792f10b234bc9aa7b3189fbbf1ea4
I20260806 02:45:24.531114 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d8219e0438e46f783037537e687f945" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" is_pre_election: true
I20260806 02:45:24.531451 21710 raft_consensus.cc:2454] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:24.531565 21518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d8219e0438e46f783037537e687f945" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:24.531876 21518 raft_consensus.cc:2454] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:24.532195 21806 leader_election.cc:302] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:24.532442 22160 tablet_bootstrap.cc:654] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:24.533283 22239 leader_election.cc:288] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:24.533661 22214 raft_consensus.cc:2781] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:24.533916 22214 raft_consensus.cc:482] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:24.535301 22149 ts_tablet_manager.cc:1918] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:24.539559 22214 raft_consensus.cc:3037] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:24.542354 22091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:45:24.542523 22157 ts_tablet_manager.cc:1939] T cba792f10b234bc9aa7b3189fbbf1ea4 P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:24.543989 22091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:45:24.544005 21064 catalog_manager.cc:4841] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): tablet cba792f10b234bc9aa7b3189fbbf1ea4 (table default.table_c94f1c901de74f71bc21408cd63fa3ea [id=2981a18bee6a4705ae5547688e2728f8]) successfully deleted
I20260806 02:45:24.544557 22091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:45:24.545789 22164 tablet_bootstrap.cc:654] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:24.538610 22052 consensus_queue.cc:1035] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:24.545991 22214 raft_consensus.cc:504] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.554677 22149 log.cc:1196] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/d0956f9352c84a069760fa70da9e8881
I20260806 02:45:24.555418 22149 ts_tablet_manager.cc:1939] T d0956f9352c84a069760fa70da9e8881 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:24.556088 21518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d8219e0438e46f783037537e687f945" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:24.556260 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d8219e0438e46f783037537e687f945" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
I20260806 02:45:24.556475 21710 raft_consensus.cc:3037] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:24.556538 21518 raft_consensus.cc:3037] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:24.556766 22214 leader_election.cc:288] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:24.540385 21077 catalog_manager.cc:3411] renamed table default.table_54a040201f184199874cbda298addd03 [id=4c9f8d8c878a4b128b71e594b536267e] in HMS: new name default.table_ddb47e155c91465f8e38147105f2a6bd
I20260806 02:45:24.558955 22149 tablet_replica.cc:330] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:24.559322 22149 raft_consensus.cc:2229] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:24.559767 22149 raft_consensus.cc:2258] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:24.560093 21518 raft_consensus.cc:2454] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:24.560750 21803 leader_election.cc:302] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:24.553884 22150 tablet_bootstrap.cc:654] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:24.561265 22214 raft_consensus.cc:2781] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:24.561537 22214 raft_consensus.cc:686] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [term 1 LEADER]: Becoming Leader. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Running, Role: LEADER
I20260806 02:45:24.562294 22214 consensus_queue.cc:237] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.566094 22164 tablet_bootstrap.cc:492] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:24.566462 22164 ts_tablet_manager.cc:1397] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.063s user 0.004s sys 0.002s
I20260806 02:45:24.568507 22164 raft_consensus.cc:348] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.568979 22164 raft_consensus.cc:374] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:24.568969 21495 tablet_service.cc:1467] Processing CreateTablet for tablet d571f1c27e944cb4874a6c30a141c2ad (DEFAULT_TABLE table=default.table_c94f1c901de74f71bc21408cd63fa3ea [id=2981a18bee6a4705ae5547688e2728f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:24.569125 22164 raft_consensus.cc:729] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:24.569490 21495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d571f1c27e944cb4874a6c30a141c2ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:24.569501 22164 consensus_queue.cc:260] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.570233 22164 ts_tablet_manager.cc:1428] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 02:45:24.570715 22164 tablet_bootstrap.cc:492] T 3d44c58ba1f044b38f10da10cec410e3 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:24.570847 21068 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet d0956f9352c84a069760fa70da9e8881 (table default.table_910edc3d61454438a69af3c32d9b721f [id=1bbd1184e7754c5bb9ffdf82791c73fb]) successfully deleted
I20260806 02:45:24.577431 21710 raft_consensus.cc:2454] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:24.578262 22149 ts_tablet_manager.cc:1905] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:24.579167 22160 tablet_bootstrap.cc:492] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:24.579214 22100 consensus_queue.cc:1035] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260806 02:45:24.580078 22164 tablet_bootstrap.cc:654] T 3d44c58ba1f044b38f10da10cec410e3 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:24.580853 22160 ts_tablet_manager.cc:1397] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.077s user 0.006s sys 0.000s
I20260806 02:45:24.597848 22150 tablet_bootstrap.cc:492] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:24.598249 22150 ts_tablet_manager.cc:1397] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.099s user 0.005s sys 0.002s
I20260806 02:45:24.600193 22160 raft_consensus.cc:348] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.600606 22160 raft_consensus.cc:374] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:24.600649 22150 raft_consensus.cc:348] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.601022 22150 raft_consensus.cc:374] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:24.601114 22150 raft_consensus.cc:729] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:24.601244 22160 raft_consensus.cc:729] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:24.601491 22150 consensus_queue.cc:260] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.602200 22150 ts_tablet_manager.cc:1428] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 02:45:24.606772 22160 consensus_queue.cc:260] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.612390 22160 ts_tablet_manager.cc:1428] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.031s user 0.000s sys 0.003s
I20260806 02:45:24.616330 22160 tablet_bootstrap.cc:492] T 3d44c58ba1f044b38f10da10cec410e3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:24.621094 21081 catalog_manager.cc:5495] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3). New cstate: current_term: 1 leader_uuid: "8ebd96b903b944f3b06851324b737bd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: HEALTHY } } }
I20260806 02:45:24.622488 21081 catalog_manager.cc:5587] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883) does not have the latest schema for tablet 2d8219e0438e46f783037537e687f945 (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]). Expected version 1 got 0
I20260806 02:45:24.626732 22150 tablet_bootstrap.cc:492] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:24.637009 22164 tablet_bootstrap.cc:492] T 3d44c58ba1f044b38f10da10cec410e3 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:24.637253 22164 ts_tablet_manager.cc:1397] T 3d44c58ba1f044b38f10da10cec410e3 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.067s user 0.007s sys 0.000s
I20260806 02:45:24.638445 22164 raft_consensus.cc:348] T 3d44c58ba1f044b38f10da10cec410e3 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.638775 22164 raft_consensus.cc:374] T 3d44c58ba1f044b38f10da10cec410e3 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:24.638814 22150 tablet_bootstrap.cc:654] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:24.638885 22164 raft_consensus.cc:729] T 3d44c58ba1f044b38f10da10cec410e3 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:24.639243 22164 consensus_queue.cc:260] T 3d44c58ba1f044b38f10da10cec410e3 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.640172 22164 ts_tablet_manager.cc:1428] T 3d44c58ba1f044b38f10da10cec410e3 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 02:45:24.640456 22164 tablet_bootstrap.cc:492] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:24.642843 22160 tablet_bootstrap.cc:654] T 3d44c58ba1f044b38f10da10cec410e3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:24.647768 22153 ts_tablet_manager.cc:1918] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:24.648015 22153 log.cc:1196] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/fde40ab726344932997271f125940c61
I20260806 02:45:24.648680 22153 ts_tablet_manager.cc:1939] T fde40ab726344932997271f125940c61 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:24.652100 22149 ts_tablet_manager.cc:1918] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:24.654196 22164 tablet_bootstrap.cc:654] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:24.656694 22149 log.cc:1196] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/cba792f10b234bc9aa7b3189fbbf1ea4
I20260806 02:45:24.657939 22160 tablet_bootstrap.cc:492] T 3d44c58ba1f044b38f10da10cec410e3 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:24.658982 22160 ts_tablet_manager.cc:1397] T 3d44c58ba1f044b38f10da10cec410e3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.043s user 0.003s sys 0.002s
I20260806 02:45:24.659883 22149 ts_tablet_manager.cc:1939] T cba792f10b234bc9aa7b3189fbbf1ea4 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:24.662685 21068 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet cba792f10b234bc9aa7b3189fbbf1ea4 (table default.table_c94f1c901de74f71bc21408cd63fa3ea [id=2981a18bee6a4705ae5547688e2728f8]) successfully deleted
I20260806 02:45:24.663278 22160 raft_consensus.cc:348] T 3d44c58ba1f044b38f10da10cec410e3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.663684 22160 raft_consensus.cc:374] T 3d44c58ba1f044b38f10da10cec410e3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:24.664431 22153 tablet_replica.cc:330] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:24.664690 22153 raft_consensus.cc:2229] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:24.664938 22153 raft_consensus.cc:2258] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:24.664968 22150 tablet_bootstrap.cc:492] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:24.663841 22160 raft_consensus.cc:729] T 3d44c58ba1f044b38f10da10cec410e3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:24.665112 21064 catalog_manager.cc:4841] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): tablet fde40ab726344932997271f125940c61 (table default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443]) successfully deleted
I20260806 02:45:24.665385 22150 ts_tablet_manager.cc:1397] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.039s user 0.008s sys 0.001s
I20260806 02:45:24.666371 22160 consensus_queue.cc:260] T 3d44c58ba1f044b38f10da10cec410e3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.667237 22150 raft_consensus.cc:348] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.667460 22153 ts_tablet_manager.cc:1905] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:24.667601 22150 raft_consensus.cc:374] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:24.667866 22150 raft_consensus.cc:729] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:24.668325 22150 consensus_queue.cc:260] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.669458 22150 ts_tablet_manager.cc:1428] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260806 02:45:24.669834 22150 tablet_bootstrap.cc:492] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:24.672672 22222 raft_consensus.cc:482] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:24.672992 22222 raft_consensus.cc:504] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.673918 22222 leader_election.cc:288] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:24.674661 21521 raft_consensus.cc:1261] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Refusing update from remote peer 8ebd96b903b944f3b06851324b737bd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:45:24.674851 21519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d44c58ba1f044b38f10da10cec410e3" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:24.674953 21710 raft_consensus.cc:1261] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Refusing update from remote peer 8ebd96b903b944f3b06851324b737bd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:45:24.675177 21519 raft_consensus.cc:2454] T 3d44c58ba1f044b38f10da10cec410e3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:24.675565 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d44c58ba1f044b38f10da10cec410e3" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:24.675917 21888 raft_consensus.cc:2454] T 3d44c58ba1f044b38f10da10cec410e3 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:24.675572 22239 consensus_queue.cc:1035] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 02:45:24.676895 22214 consensus_queue.cc:1035] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:24.677137 22150 tablet_bootstrap.cc:654] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:24.680907 22160 ts_tablet_manager.cc:1428] T 3d44c58ba1f044b38f10da10cec410e3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.021s user 0.003s sys 0.000s
I20260806 02:45:24.681645 21617 leader_election.cc:302] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f; no voters:
I20260806 02:45:24.682332 22052 raft_consensus.cc:2781] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:24.682627 22052 raft_consensus.cc:482] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:24.682842 22052 raft_consensus.cc:3037] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:24.691546 22150 tablet_bootstrap.cc:492] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:24.691823 22150 ts_tablet_manager.cc:1397] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.003s
I20260806 02:45:24.692648 22052 raft_consensus.cc:504] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.693218 22150 raft_consensus.cc:348] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.693682 22150 raft_consensus.cc:374] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:24.693850 22150 raft_consensus.cc:729] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:24.693879 21521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d44c58ba1f044b38f10da10cec410e3" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:24.694235 21521 raft_consensus.cc:3037] T 3d44c58ba1f044b38f10da10cec410e3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:24.694339 22150 consensus_queue.cc:260] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.694571 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d44c58ba1f044b38f10da10cec410e3" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:24.694847 21888 raft_consensus.cc:3037] T 3d44c58ba1f044b38f10da10cec410e3 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:24.696064 22150 ts_tablet_manager.cc:1428] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 02:45:24.695084 21081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58094:
table { table_name: "default.Table_c2d5cb4cc8a44eaca15ba851ecc366c8" } modify_external_catalogs: true
I20260806 02:45:24.696652 21081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58094:
table { table_name: "default.Table_c2d5cb4cc8a44eaca15ba851ecc366c8" } modify_external_catalogs: true
I20260806 02:45:24.698161 21521 raft_consensus.cc:2454] T 3d44c58ba1f044b38f10da10cec410e3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:24.698846 21617 leader_election.cc:302] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f; no voters:
I20260806 02:45:24.699224 22222 raft_consensus.cc:2781] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:24.699369 21888 raft_consensus.cc:2454] T 3d44c58ba1f044b38f10da10cec410e3 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:24.699425 22222 raft_consensus.cc:686] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Becoming Leader. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Running, Role: LEADER
I20260806 02:45:24.699821 22222 consensus_queue.cc:237] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.700213 22160 tablet_bootstrap.cc:492] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:24.707809 22052 leader_election.cc:288] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:24.708989 22164 tablet_bootstrap.cc:492] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:24.709358 22164 ts_tablet_manager.cc:1397] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.069s user 0.006s sys 0.000s
I20260806 02:45:24.710346 22164 raft_consensus.cc:348] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.710918 22164 raft_consensus.cc:374] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:24.711098 22164 raft_consensus.cc:729] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:24.712519 22160 tablet_bootstrap.cc:654] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:24.713013 22164 consensus_queue.cc:260] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.721227 22164 ts_tablet_manager.cc:1428] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.012s user 0.000s sys 0.002s
I20260806 02:45:24.729024 22214 raft_consensus.cc:482] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:24.729274 22214 raft_consensus.cc:504] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.729930 22214 leader_election.cc:288] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:24.733757 21705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4e3f2c8eb2c4ea8a1b041be073bdd08" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" is_pre_election: true
I20260806 02:45:24.734186 21705 raft_consensus.cc:2454] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:24.736395 21806 leader_election.cc:302] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:24.737056 22239 raft_consensus.cc:2781] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:24.737250 22239 raft_consensus.cc:482] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:24.737489 22239 raft_consensus.cc:3037] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:24.741039 22239 raft_consensus.cc:504] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.743108 22175 mvcc.cc:205] Tried to move back new op lower bound from 7315391793844502528 to 7315391793410371584. Current Snapshot: MvccSnapshot[applied={T|T < 7315391793844502528}]
I20260806 02:45:24.744699 22174 mvcc.cc:205] Tried to move back new op lower bound from 7315391793844502528 to 7315391793410371584. Current Snapshot: MvccSnapshot[applied={T|T < 7315391793844502528}]
I20260806 02:45:24.751031 22285 tablet.cc:1722] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:24.752628 22288 tablet.cc:1722] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:24.755324 22239 leader_election.cc:288] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:24.756937 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4e3f2c8eb2c4ea8a1b041be073bdd08" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
I20260806 02:45:24.760241 21710 raft_consensus.cc:3037] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:24.761427 21521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4e3f2c8eb2c4ea8a1b041be073bdd08" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:24.761778 21521 raft_consensus.cc:2454] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:24.763060 21521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4e3f2c8eb2c4ea8a1b041be073bdd08" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:24.763257 21521 raft_consensus.cc:3037] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:24.766165 22222 raft_consensus.cc:482] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:24.766480 22222 raft_consensus.cc:504] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.767464 21521 raft_consensus.cc:2454] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:24.767515 22222 leader_election.cc:288] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:24.767870 21803 leader_election.cc:302] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:24.768231 22239 raft_consensus.cc:2781] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:24.768507 21518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3196f6ee70d34d298b7db56d9759fab7" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:24.768985 21518 raft_consensus.cc:2454] T 3196f6ee70d34d298b7db56d9759fab7 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:24.768882 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3196f6ee70d34d298b7db56d9759fab7" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:24.769194 21888 raft_consensus.cc:2454] T 3196f6ee70d34d298b7db56d9759fab7 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:24.769589 21617 leader_election.cc:302] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f; no voters:
I20260806 02:45:24.769954 22222 raft_consensus.cc:2781] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:24.770205 22222 raft_consensus.cc:482] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:24.770388 22222 raft_consensus.cc:3037] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:24.773243 21710 raft_consensus.cc:2454] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:24.775353 22222 raft_consensus.cc:504] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.776647 21518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3196f6ee70d34d298b7db56d9759fab7" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:24.777019 21518 raft_consensus.cc:3037] T 3196f6ee70d34d298b7db56d9759fab7 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:24.780305 21518 raft_consensus.cc:2454] T 3196f6ee70d34d298b7db56d9759fab7 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:24.780889 21617 leader_election.cc:302] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f; no voters:
I20260806 02:45:24.781319 22052 raft_consensus.cc:2781] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:24.781674 22052 raft_consensus.cc:686] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Becoming Leader. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Running, Role: LEADER
I20260806 02:45:24.782051 22052 consensus_queue.cc:237] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.786548 22239 raft_consensus.cc:686] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [term 1 LEADER]: Becoming Leader. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Running, Role: LEADER
I20260806 02:45:24.786955 22239 consensus_queue.cc:237] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.789177 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3196f6ee70d34d298b7db56d9759fab7" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:24.789563 21888 raft_consensus.cc:3037] T 3196f6ee70d34d298b7db56d9759fab7 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
W20260806 02:45:24.791380 22281 log.cc:924] Time spent T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5: Append to log took a long time: real 0.063s user 0.000s sys 0.000s
I20260806 02:45:24.791939 22153 ts_tablet_manager.cc:1918] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:24.792119 22153 log.cc:1196] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/cba792f10b234bc9aa7b3189fbbf1ea4
I20260806 02:45:24.792652 22153 ts_tablet_manager.cc:1939] T cba792f10b234bc9aa7b3189fbbf1ea4 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:24.796190 22156 mvcc.cc:205] Tried to move back new op lower bound from 7315391793844502528 to 7315391793410371584. Current Snapshot: MvccSnapshot[applied={T|T < 7315391793844502528}]
I20260806 02:45:24.796705 22292 tablet.cc:1722] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:24.798461 22222 leader_election.cc:288] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:24.802166 21888 raft_consensus.cc:2454] T 3196f6ee70d34d298b7db56d9759fab7 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:24.802325 21064 catalog_manager.cc:4841] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): tablet cba792f10b234bc9aa7b3189fbbf1ea4 (table default.table_c94f1c901de74f71bc21408cd63fa3ea [id=2981a18bee6a4705ae5547688e2728f8]) successfully deleted
I20260806 02:45:24.812817 22160 tablet_bootstrap.cc:492] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:24.813169 22160 ts_tablet_manager.cc:1397] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.113s user 0.004s sys 0.008s
I20260806 02:45:24.814606 22160 raft_consensus.cc:348] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.815055 22160 raft_consensus.cc:374] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:24.815199 22160 raft_consensus.cc:729] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:24.815605 22160 consensus_queue.cc:260] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.821281 22160 ts_tablet_manager.cc:1428] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.008s user 0.000s sys 0.004s
I20260806 02:45:24.835421 22239 raft_consensus.cc:482] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:24.835605 22239 raft_consensus.cc:504] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.836596 22239 leader_election.cc:288] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:24.837924 21519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d571f1c27e944cb4874a6c30a141c2ad" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:24.838239 21519 raft_consensus.cc:2454] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:24.838779 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d571f1c27e944cb4874a6c30a141c2ad" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" is_pre_election: true
I20260806 02:45:24.839118 21710 raft_consensus.cc:2454] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:24.839856 21806 leader_election.cc:302] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:24.840632 22239 raft_consensus.cc:2781] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:24.840806 22239 raft_consensus.cc:482] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:24.840992 22239 raft_consensus.cc:3037] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:24.845047 22239 raft_consensus.cc:504] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.846969 21519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d571f1c27e944cb4874a6c30a141c2ad" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:24.847323 21519 raft_consensus.cc:3037] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:24.851043 21519 raft_consensus.cc:2454] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:24.851672 21803 leader_election.cc:302] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:24.852090 22234 raft_consensus.cc:2781] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:24.852567 22234 raft_consensus.cc:686] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [term 1 LEADER]: Becoming Leader. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Running, Role: LEADER
I20260806 02:45:24.853183 22234 consensus_queue.cc:237] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:24.859256 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d571f1c27e944cb4874a6c30a141c2ad" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
I20260806 02:45:24.859716 21710 raft_consensus.cc:3037] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:24.864245 21710 raft_consensus.cc:2454] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:24.865259 22239 leader_election.cc:288] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
W20260806 02:45:24.882316 21064 consensus_peers.cc:487] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a -> Peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Couldn't send request to peer bfdbf4ef60014ea98bbcf9b6a9033f22. Status: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260806 02:45:25.093828 22091 rpc_server.cc:307] RPC server started. Bound to: 127.20.60.61:36797
I20260806 02:45:25.096436 22338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.60.61:36797 every 8 connection(s)
I20260806 02:45:25.096737 22091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data/info.pb
I20260806 02:45:25.105463 20720 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu as pid 22091
I20260806 02:45:25.108320 22340 sys_catalog.cc:263] Verifying existing consensus state
I20260806 02:45:25.127024 22340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Bootstrap starting.
I20260806 02:45:25.143260 22340 log.cc:824] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Log is configured to *not* fsync() on all Append() calls
I20260806 02:45:25.147780 22344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a225e002d607451a94b3686f10aeea9a)
I20260806 02:45:25.148221 22344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:25.154382 21096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "894be089543b4aa29b3edfcc5f3085e9" candidate_term: 4 candidate_status { last_received { term: 3 index: 48 } } ignore_live_leader: false dest_uuid: "a225e002d607451a94b3686f10aeea9a" is_pre_election: true
I20260806 02:45:25.156180 22344 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
I20260806 02:45:25.176628 22340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=1 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 02:45:25.178390 22340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Bootstrap complete.
I20260806 02:45:25.200448 22340 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:25.202488 22340 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfdbf4ef60014ea98bbcf9b6a9033f22, State: Initialized, Role: FOLLOWER
I20260806 02:45:25.203814 22340 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:25.211825 22347 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:25.212591 22347 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:25.216477 22340 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:45:25.216727 22304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "894be089543b4aa29b3edfcc5f3085e9" candidate_term: 4 candidate_status { last_received { term: 3 index: 48 } } ignore_live_leader: false dest_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" is_pre_election: true
I20260806 02:45:25.217703 22304 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 894be089543b4aa29b3edfcc5f3085e9 in term 3.
I20260806 02:45:25.218706 21251 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9, bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: a225e002d607451a94b3686f10aeea9a
I20260806 02:45:25.219238 22344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260806 02:45:25.219497 22344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a225e002d607451a94b3686f10aeea9a)
I20260806 02:45:25.219621 22344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 3 FOLLOWER]: Advancing to term 4
I20260806 02:45:25.223134 22344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:25.224604 21096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "894be089543b4aa29b3edfcc5f3085e9" candidate_term: 4 candidate_status { last_received { term: 3 index: 48 } } ignore_live_leader: false dest_uuid: "a225e002d607451a94b3686f10aeea9a"
I20260806 02:45:25.224772 22344 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 4 election: Requested vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
I20260806 02:45:25.224952 22304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "894be089543b4aa29b3edfcc5f3085e9" candidate_term: 4 candidate_status { last_received { term: 3 index: 48 } } ignore_live_leader: false dest_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22"
I20260806 02:45:25.227339 22304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 3 FOLLOWER]: Advancing to term 4
I20260806 02:45:25.236848 22304 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 894be089543b4aa29b3edfcc5f3085e9 in term 4.
I20260806 02:45:25.237519 21251 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9, bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: a225e002d607451a94b3686f10aeea9a
I20260806 02:45:25.238027 22344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 4 FOLLOWER]: Leader election won for term 4
I20260806 02:45:25.238793 22344 raft_consensus.cc:686] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 4 LEADER]: Becoming Leader. State: Replica: 894be089543b4aa29b3edfcc5f3085e9, State: Running, Role: LEADER
I20260806 02:45:25.239169 22344 consensus_queue.cc:237] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 48, Committed index: 48, Last appended: 3.48, Last appended by leader: 48, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:25.240450 22361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 894be089543b4aa29b3edfcc5f3085e9. Latest consensus state: current_term: 4 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:25.241479 22361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:25.248709 22363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:45:25.252722 22363 catalog_manager.cc:669] Loaded metadata for table default.table_910edc3d61454438a69af3c32d9b721f [id=1bbd1184e7754c5bb9ffdf82791c73fb]
I20260806 02:45:25.259434 22363 catalog_manager.cc:669] Loaded metadata for table default.table_a7715d8e001f48b2899c3c2a56a18a06 [id=21d847eb419e4c0998a01f87c8cd92da]
I20260806 02:45:25.260512 22363 catalog_manager.cc:669] Loaded metadata for table default.table_c94f1c901de74f71bc21408cd63fa3ea [id=2981a18bee6a4705ae5547688e2728f8]
I20260806 02:45:25.261796 22363 catalog_manager.cc:669] Loaded metadata for table default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443]
I20260806 02:45:25.262466 22363 catalog_manager.cc:669] Loaded metadata for table default.table_ea1a38afd92a4b15bfabb80e58f8c644 [id=383ba93bcba140c1b1bd2b36f8720b86]
I20260806 02:45:25.263029 22363 catalog_manager.cc:669] Loaded metadata for table default.table_54a040201f184199874cbda298addd03 [id=4c9f8d8c878a4b128b71e594b536267e]
I20260806 02:45:25.263659 22363 catalog_manager.cc:669] Loaded metadata for table default.table_00f2c397972f4d8e828d61d7821abc00 [id=610932bdd3f94b4e886d586c28ff6249]
I20260806 02:45:25.265143 22363 catalog_manager.cc:669] Loaded metadata for table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]
I20260806 02:45:25.265738 22363 catalog_manager.cc:669] Loaded metadata for table default.table_00c06d2b7d6d40658a4390f4ee3bbab5 [id=c6a05485b0a54967ba8904aa969edd3f]
I20260806 02:45:25.266474 22363 catalog_manager.cc:669] Loaded metadata for table default.table_c2d5cb4cc8a44eaca15ba851ecc366c8 [id=ccf2db5226644b498abe88bba95d6fef]
I20260806 02:45:25.267218 22363 catalog_manager.cc:669] Loaded metadata for table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e]
I20260806 02:45:25.267910 22363 catalog_manager.cc:669] Loaded metadata for table default.table_f36563aa9b94423ebebd9741529ba14c [id=f9ea378b10f94d99a856cf000c426373]
I20260806 02:45:25.271811 22363 tablet_loader.cc:96] loaded metadata for tablet 043839a5eca143d5882f40faaab39692 (table default.table_00f2c397972f4d8e828d61d7821abc00 [id=610932bdd3f94b4e886d586c28ff6249])
I20260806 02:45:25.272571 22363 tablet_loader.cc:96] loaded metadata for tablet 237439c720344cfd97bd04bf8368a375 (table default.table_ea1a38afd92a4b15bfabb80e58f8c644 [id=383ba93bcba140c1b1bd2b36f8720b86])
I20260806 02:45:25.273146 22363 tablet_loader.cc:96] loaded metadata for tablet 2d8219e0438e46f783037537e687f945 (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022])
I20260806 02:45:25.273712 22363 tablet_loader.cc:96] loaded metadata for tablet 3196f6ee70d34d298b7db56d9759fab7 (table default.table_00c06d2b7d6d40658a4390f4ee3bbab5 [id=c6a05485b0a54967ba8904aa969edd3f])
I20260806 02:45:25.274250 22363 tablet_loader.cc:96] loaded metadata for tablet 3b3a4c79185241a7a8522135f0a513b3 (table default.table_c2d5cb4cc8a44eaca15ba851ecc366c8 [id=ccf2db5226644b498abe88bba95d6fef])
I20260806 02:45:25.274868 22363 tablet_loader.cc:96] loaded metadata for tablet 3d44c58ba1f044b38f10da10cec410e3 (table default.table_910edc3d61454438a69af3c32d9b721f [id=1bbd1184e7754c5bb9ffdf82791c73fb])
I20260806 02:45:25.275477 22363 tablet_loader.cc:96] loaded metadata for tablet 3f0c5aff461940d999d0ed14292db320 (table default.table_a7715d8e001f48b2899c3c2a56a18a06 [id=21d847eb419e4c0998a01f87c8cd92da])
I20260806 02:45:25.276142 22363 tablet_loader.cc:96] loaded metadata for tablet 6061f59aedec4e0a9b96239ebe6fa0df (table default.table_54a040201f184199874cbda298addd03 [id=4c9f8d8c878a4b128b71e594b536267e])
I20260806 02:45:25.277160 22363 tablet_loader.cc:96] loaded metadata for tablet b4e3f2c8eb2c4ea8a1b041be073bdd08 (table default.table_358a74d95709455e931a3b3c82ab656e [id=370d5895c35f4c96878270e71cac2443])
I20260806 02:45:25.278782 22363 tablet_loader.cc:96] loaded metadata for tablet d25a9c5a0123446dac00ad895381f96a (table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e])
I20260806 02:45:25.279249 22363 tablet_loader.cc:96] loaded metadata for tablet d571f1c27e944cb4874a6c30a141c2ad (table default.table_c94f1c901de74f71bc21408cd63fa3ea [id=2981a18bee6a4705ae5547688e2728f8])
I20260806 02:45:25.279768 22363 tablet_loader.cc:96] loaded metadata for tablet d8c3b721c7bd4c9b81ef7744c1230599 (table default.table_f36563aa9b94423ebebd9741529ba14c [id=f9ea378b10f94d99a856cf000c426373])
I20260806 02:45:25.280797 22363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:45:25.280987 21285 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 4 LEADER]: Rejecting Update request from peer a225e002d607451a94b3686f10aeea9a for earlier term 3. Current term is 4. Ops: []
I20260806 02:45:25.282195 21962 consensus_queue.cc:1046] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 }, Status: INVALID_TERM, Last received: 3.48, Next index: 49, Last known committed idx: 48, Time since last communication: 0.000s
I20260806 02:45:25.285156 21962 consensus_queue.cc:1225] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [LEADER]: Peer 894be089543b4aa29b3edfcc5f3085e9 log is divergent from this leader: its last log entry 4.49 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 48
I20260806 02:45:25.285872 22371 raft_consensus.cc:3032] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 3 LEADER]: Stepping down as leader of term 3
I20260806 02:45:25.286115 22371 raft_consensus.cc:729] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a225e002d607451a94b3686f10aeea9a, State: Running, Role: LEADER
I20260806 02:45:25.286564 22371 consensus_queue.cc:260] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 48, Committed index: 48, Last appended: 3.48, Last appended by leader: 48, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:25.286989 22363 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:25.287180 22363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:45:25.287303 22371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 3 FOLLOWER]: Advancing to term 4
I20260806 02:45:25.290266 22363 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:45:25.292533 22363 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Loaded TSK: 0
I20260806 02:45:25.293565 22363 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:45:25.295511 22363 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260806 02:45:25.297658 22363 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: 11
I20260806 02:45:25.312042 21581 heartbeater.cc:503] Master 127.20.60.60:42309 was elected leader, sending a full tablet report...
I20260806 02:45:25.312326 22370 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:25.312501 22370 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: loading cluster ID for follower catalog manager: success
I20260806 02:45:25.320802 22370 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: acquiring CA information for follower catalog manager: success
I20260806 02:45:25.327577 21583 heartbeater.cc:347] Connected to a master server at 127.20.60.61:36797
I20260806 02:45:25.336824 22370 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 02:45:25.343689 21096 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 4 FOLLOWER]: Refusing update from remote peer 894be089543b4aa29b3edfcc5f3085e9: Log matching property violated. Preceding OpId in replica: term: 3 index: 48. Preceding OpId from leader: term: 4 index: 50. (index mismatch)
I20260806 02:45:25.344359 22304 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 4 FOLLOWER]: Refusing update from remote peer 894be089543b4aa29b3edfcc5f3085e9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 50. (index mismatch)
I20260806 02:45:25.344671 22277 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" instance_seqno: 1785984319928949) as {username='slave'} at 127.20.60.1:53691; Asking this server to re-register.
I20260806 02:45:25.345685 22361 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 02:45:25.346019 21583 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:25.349002 22275 ts_manager.cc:194] Registered new tserver with Master: 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:25.353992 22361 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 48, Time since last communication: 0.000s
I20260806 02:45:25.356575 21761 heartbeater.cc:503] Master 127.20.60.60:42309 was elected leader, sending a full tablet report...
I20260806 02:45:25.362336 21273 catalog_manager.cc:5495] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f reported cstate change: term changed from 0 to 1, leader changed from <none> to 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2). New cstate: current_term: 1 leader_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260806 02:45:25.363898 21273 catalog_manager.cc:5495] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f reported cstate change: term changed from 0 to 1, leader changed from <none> to 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2). New cstate: current_term: 1 leader_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260806 02:45:25.374696 21763 heartbeater.cc:347] Connected to a master server at 127.20.60.61:36797
I20260806 02:45:25.391273 22371 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 894be089543b4aa29b3edfcc5f3085e9. Latest consensus state: current_term: 4 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:25.391824 22371 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:25.394454 22377 mvcc.cc:205] Tried to move back new op lower bound from 7315391796564242432 to 7315391796182401024. Current Snapshot: MvccSnapshot[applied={T|T < 7315391796564242432}]
I20260806 02:45:25.397325 22375 consensus_queue.cc:1035] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:25.399377 21950 heartbeater.cc:503] Master 127.20.60.60:42309 was elected leader, sending a full tablet report...
I20260806 02:45:25.404189 22375 consensus_queue.cc:1035] T 237439c720344cfd97bd04bf8368a375 P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:25.441435 22361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:25.441875 22361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:25.458184 22371 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:25.458494 22371 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:25.464871 21272 catalog_manager.cc:5495] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3). New cstate: current_term: 1 leader_uuid: "8ebd96b903b944f3b06851324b737bd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: HEALTHY } } }
I20260806 02:45:25.467235 21272 catalog_manager.cc:5495] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3). New cstate: current_term: 1 leader_uuid: "8ebd96b903b944f3b06851324b737bd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: HEALTHY } } }
I20260806 02:45:25.468685 21274 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022] alter complete (version 1)
I20260806 02:45:25.482177 21952 heartbeater.cc:347] Connected to a master server at 127.20.60.61:36797
I20260806 02:45:25.523607 22397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:25.524248 22397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:25.524832 22277 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" instance_seqno: 1785984320818216) as {username='slave'} at 127.20.60.2:48953; Asking this server to re-register.
I20260806 02:45:25.525986 21763 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:25.531589 22274 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" instance_seqno: 1785984321670316) as {username='slave'} at 127.20.60.3:33015; Asking this server to re-register.
I20260806 02:45:25.532794 21952 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:25.538466 22277 ts_manager.cc:194] Registered new tserver with Master: 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:25.542446 22347 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 894be089543b4aa29b3edfcc5f3085e9. Latest consensus state: current_term: 4 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:25.544615 22347 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:25.556219 22347 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:25.556612 22347 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:25.557554 22277 ts_manager.cc:194] Registered new tserver with Master: 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:25.564162 22379 consensus_queue.cc:1035] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:25.589916 22379 consensus_queue.cc:1035] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:25.634117 22385 mvcc.cc:205] Tried to move back new op lower bound from 7315391796564242432 to 7315391796182401024. Current Snapshot: MvccSnapshot[applied={T|T < 7315391796564242432 or (T in {7315391796564242432})}]
W20260806 02:45:25.642838 21387 hms_client.cc:305] Time spent drop HMS table: real 1.103s user 0.001s sys 0.000s
I20260806 02:45:25.643172 21079 catalog_manager.cc:2685] deleted HMS catalog entry for table default.table_c94f1c901de74f71bc21408cd63fa3ea [id=2981a18bee6a4705ae5547688e2728f8]
I20260806 02:45:25.672189 22234 consensus_queue.cc:1035] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:25.683145 21076 catalog_manager.cc:2148] created HMS catalog entry for table [id=b4cb7469fede4d7cb3ae9bc653f5eec9]
W20260806 02:45:25.684204 21076 catalog_manager.cc:2176] Illegal state: an error occurred while writing to the sys-catalog: replica a225e002d607451a94b3686f10aeea9a is not leader of this config: current role FOLLOWER
I20260806 02:45:25.703243 22414 consensus_queue.cc:1035] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:25.757436 22414 consensus_queue.cc:1035] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:25.763986 21518 raft_consensus.cc:1261] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Refusing update from remote peer 8ebd96b903b944f3b06851324b737bd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:45:25.765165 22412 consensus_queue.cc:1035] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:25.773272 22426 tablet.cc:1722] T 3d44c58ba1f044b38f10da10cec410e3 P 88dc3ab284a64961b1fbdb5c53d1267f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:25.778378 21345 catalog_manager.cc:2709] Deleting table default.table_c94f1c901de74f71bc21408cd63fa3ea [id=2981a18bee6a4705ae5547688e2728f8] in response to Hive Metastore notification log event 18
I20260806 02:45:25.787300 22438 tablet.cc:1722] T 3d44c58ba1f044b38f10da10cec410e3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:25.790303 22437 tablet.cc:1722] T 043839a5eca143d5882f40faaab39692 P 88dc3ab284a64961b1fbdb5c53d1267f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:25.794167 22442 tablet.cc:1722] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:25.797098 21273 catalog_manager.cc:5587] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655) does not have the latest schema for tablet 3d44c58ba1f044b38f10da10cec410e3 (table default.table_e41f56f2e4a1439ca6b8f2bca460a670 [id=1bbd1184e7754c5bb9ffdf82791c73fb]). Expected version 1 got 0
I20260806 02:45:25.799276 21345 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Sending DeleteTablet for 3 replicas of tablet d571f1c27e944cb4874a6c30a141c2ad
I20260806 02:45:25.774809 22428 tablet.cc:1722] T 3d44c58ba1f044b38f10da10cec410e3 P 8ebd96b903b944f3b06851324b737bd5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:25.802543 21272 catalog_manager.cc:5587] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593) does not have the latest schema for tablet b4e3f2c8eb2c4ea8a1b041be073bdd08 (table default.table_8797f6f61c444bd0a4e4555f6ca2fe30 [id=370d5895c35f4c96878270e71cac2443]). Expected version 1 got 0
I20260806 02:45:25.803463 21681 tablet_service.cc:1514] Processing DeleteTablet for tablet d571f1c27e944cb4874a6c30a141c2ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:45:25 UTC) from {username='slave'} at 127.0.0.1:38406
I20260806 02:45:25.812680 22438 tablet.cc:1722] T 043839a5eca143d5882f40faaab39692 P 48b0c1e2106b4d9b941929ddc6d97dd5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:25.814807 21863 tablet_service.cc:1514] Processing DeleteTablet for tablet d571f1c27e944cb4874a6c30a141c2ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:45:25 UTC) from {username='slave'} at 127.0.0.1:56374
I20260806 02:45:25.832399 22428 tablet.cc:1722] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:25.832463 21273 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: default.table_e41f56f2e4a1439ca6b8f2bca460a670 [id=1bbd1184e7754c5bb9ffdf82791c73fb] alter complete (version 1)
I20260806 02:45:25.836534 22438 tablet.cc:1722] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:25.842145 21272 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: default.table_8797f6f61c444bd0a4e4555f6ca2fe30 [id=370d5895c35f4c96878270e71cac2443] alter complete (version 1)
I20260806 02:45:25.850101 21272 catalog_manager.cc:5587] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593) does not have the latest schema for tablet b4e3f2c8eb2c4ea8a1b041be073bdd08 (table default.table_8797f6f61c444bd0a4e4555f6ca2fe30 [id=370d5895c35f4c96878270e71cac2443]). Expected version 1 got 0
I20260806 02:45:25.851239 22445 tablet_replica.cc:330] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:25.851639 22445 raft_consensus.cc:2229] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:45:25.852088 22445 raft_consensus.cc:2258] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:25.853077 22445 ts_tablet_manager.cc:1905] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:25.853631 22436 tablet.cc:1722] T 043839a5eca143d5882f40faaab39692 P 8ebd96b903b944f3b06851324b737bd5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:25.853842 22444 tablet_replica.cc:330] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:25.854246 22444 raft_consensus.cc:2229] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:25.854667 22444 raft_consensus.cc:2258] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:25.854782 21073 catalog_manager.cc:2148] created HMS catalog entry for table [id=fe86c442832f49bc8efdb723bab0fc10]
W20260806 02:45:25.855852 21073 catalog_manager.cc:2176] Illegal state: an error occurred while writing to the sys-catalog: replica a225e002d607451a94b3686f10aeea9a is not leader of this config: current role FOLLOWER
I20260806 02:45:25.868104 21272 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: default.table_21221d53bf9f4c46a5ffe671a1ffe424 [id=610932bdd3f94b4e886d586c28ff6249] alter complete (version 1)
I20260806 02:45:25.878535 21495 tablet_service.cc:1514] Processing DeleteTablet for tablet d571f1c27e944cb4874a6c30a141c2ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:45:25 UTC) from {username='slave'} at 127.0.0.1:33118
I20260806 02:45:25.878885 22444 ts_tablet_manager.cc:1905] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:25.888991 22445 ts_tablet_manager.cc:1918] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260806 02:45:25.889129 21391 catalog_manager.cc:3810] Illegal state: an error occurred while updating the sys-catalog: replica a225e002d607451a94b3686f10aeea9a is not leader of this config: current role FOLLOWER
W20260806 02:45:25.889643 21391 hms_notification_log_listener.cc:294] Failed to handle Hive Metastore notification: 12 ALTER_TABLE default.table_e41f56f2e4a1439ca6b8f2bca460a670: Illegal state: an error occurred while updating the sys-catalog: replica a225e002d607451a94b3686f10aeea9a is not leader of this config: current role FOLLOWER
W20260806 02:45:25.889994 21391 hms_notification_log_listener.cc:134] Hive Metastore notification log listener poll failed: Service unavailable: lost leadership while handling Hive Metastore notification log events: an error occurred while updating the sys-catalog: replica a225e002d607451a94b3686f10aeea9a is not leader of this config: current role FOLLOWER
I20260806 02:45:25.890475 21084 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:58094 (request call id 13) took 3428 ms. Trace:
I20260806 02:45:25.890645 21084 rpcz_store.cc:276] 0806 02:45:22.462047 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:22.462102 (+ 55us) service_pool.cc:227] Handling call
0806 02:45:23.530668 (+1068566us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0806 02:45:25.890356 (+2359688us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":13359,"Hive Metastore.run_cpu_time_us":811,"Hive Metastore.run_wall_time_us":112691}
W20260806 02:45:25.891435 21084 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.20.60.3:34437 (request call id 19) took 1174 ms (client timeout 1000 ms). Trace:
W20260806 02:45:25.891544 21084 rpcz_store.cc:269] 0806 02:45:24.716837 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.891036 (+1174199us) service_pool.cc:214] Skipping call since client already timed out
0806 02:45:25.891334 (+ 298us) inbound_call.cc:173] Queueing failure response
Metrics: {}
W20260806 02:45:25.891898 21084 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.20.60.2:36491 (request call id 23) took 1156 ms (client timeout 1000 ms). Trace:
I20260806 02:45:25.891973 22445 log.cc:1196] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/d571f1c27e944cb4874a6c30a141c2ad
W20260806 02:45:25.891997 21084 rpcz_store.cc:269] 0806 02:45:24.734973 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.891802 (+1156829us) service_pool.cc:214] Skipping call since client already timed out
0806 02:45:25.891842 (+ 40us) inbound_call.cc:173] Queueing failure response
Metrics: {}
W20260806 02:45:25.892325 21084 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.20.60.1:42347 (request call id 23) took 1157 ms (client timeout 1000 ms). Trace:
W20260806 02:45:25.892473 21084 rpcz_store.cc:269] 0806 02:45:24.735312 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.892226 (+1156914us) service_pool.cc:214] Skipping call since client already timed out
0806 02:45:25.892265 (+ 39us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260806 02:45:25.892879 22445 ts_tablet_manager.cc:1939] T d571f1c27e944cb4874a6c30a141c2ad P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:25.894812 22444 ts_tablet_manager.cc:1918] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260806 02:45:25.894912 21977 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.20.60.62:33227)
I20260806 02:45:25.895000 22444 log.cc:1196] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/d571f1c27e944cb4874a6c30a141c2ad
I20260806 02:45:25.896488 22444 ts_tablet_manager.cc:1939] T d571f1c27e944cb4874a6c30a141c2ad P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:25.896785 21251 catalog_manager.cc:4841] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): tablet d571f1c27e944cb4874a6c30a141c2ad (table default.table_c94f1c901de74f71bc21408cd63fa3ea [id=2981a18bee6a4705ae5547688e2728f8]) successfully deleted
I20260806 02:45:25.897821 21255 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet d571f1c27e944cb4874a6c30a141c2ad (table default.table_c94f1c901de74f71bc21408cd63fa3ea [id=2981a18bee6a4705ae5547688e2728f8]) successfully deleted
I20260806 02:45:25.900610 21273 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46462:
table { table_name: "default.Table_910edc3d61454438a69af3c32d9b721f" } new_table_name: "default.Table_e41f56f2e4a1439ca6b8f2bca460a670" modify_external_catalogs: true
W20260806 02:45:25.909301 21082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1ff877d50df44dccb8be0241b847e8c9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:45:25.911054 21083 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:58094 (request call id 12) took 3503 ms. Trace:
I20260806 02:45:25.911360 21083 rpcz_store.cc:276] 0806 02:45:22.407221 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:22.407293 (+ 72us) service_pool.cc:227] Handling call
0806 02:45:23.530668 (+1123375us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0806 02:45:25.910461 (+2379793us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0806 02:45:25.910714 (+ 253us) catalog_manager.cc:3616] Apply alter schema
0806 02:45:25.910972 (+ 258us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":126109,"Hive Metastore.run_cpu_time_us":916,"Hive Metastore.run_wall_time_us":148537,"mutex_wait_us":28,"spinlock_wait_cycles":896}
I20260806 02:45:25.912339 21083 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:58094 (request call id 34) took 3031 ms. Trace:
I20260806 02:45:25.912616 21083 rpcz_store.cc:276] 0806 02:45:22.880802 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.911728 (+3030926us) service_pool.cc:227] Handling call
0806 02:45:25.912245 (+ 517us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260806 02:45:25.917577 21977 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.20.60.60:42309)
I20260806 02:45:25.918954 21083 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:58094 (request call id 35) took 3006 ms. Trace:
I20260806 02:45:25.919262 21083 rpcz_store.cc:276] 0806 02:45:22.912676 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.913093 (+3000417us) service_pool.cc:227] Handling call
0806 02:45:25.918886 (+ 5793us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260806 02:45:25.922435 21083 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:58094 (request call id 36) took 2951 ms. Trace:
W20260806 02:45:25.923446 21977 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.20.60.60:42309)
I20260806 02:45:25.925702 21271 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46462:
table { table_name: "default.Table_f36563aa9b94423ebebd9741529ba14c" } modify_external_catalogs: true
I20260806 02:45:25.926286 21271 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46462:
table { table_name: "default.Table_f36563aa9b94423ebebd9741529ba14c" } modify_external_catalogs: true
I20260806 02:45:25.926960 21274 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46462:
table { table_name: "default.Table_8797f6f61c444bd0a4e4555f6ca2fe30" } new_table_name: "default.Table_97ee2606f8aa47dcb2c67a9ae29dd3fb" modify_external_catalogs: true
I20260806 02:45:25.927475 21272 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46462:
name: "default.Table_e8df6dde49954df7b57567fc0cd12872"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:25.926561 21080 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:58094 (request call id 15) took 3421 ms. Trace:
I20260806 02:45:25.929252 21080 rpcz_store.cc:276] 0806 02:45:22.504902 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:22.549482 (+ 44580us) service_pool.cc:227] Handling call
0806 02:45:23.530939 (+981457us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0806 02:45:25.925908 (+2394969us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0806 02:45:25.926189 (+ 281us) catalog_manager.cc:3616] Apply alter schema
0806 02:45:25.926486 (+ 297us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":274467,"Hive Metastore.run_cpu_time_us":926,"Hive Metastore.run_wall_time_us":167126}
I20260806 02:45:25.930167 21080 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:58094 (request call id 37) took 2923 ms. Trace:
I20260806 02:45:25.930311 21080 rpcz_store.cc:276] 0806 02:45:23.006801 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.929609 (+2922808us) service_pool.cc:227] Handling call
0806 02:45:25.929981 (+ 372us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260806 02:45:25.930907 21080 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:58094 (request call id 38) took 2871 ms. Trace:
I20260806 02:45:25.931002 21080 rpcz_store.cc:276] 0806 02:45:23.059216 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.930523 (+2871307us) service_pool.cc:227] Handling call
0806 02:45:25.930852 (+ 329us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260806 02:45:25.931519 21080 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:58094 (request call id 39) took 2816 ms. Trace:
I20260806 02:45:25.931655 21080 rpcz_store.cc:276] 0806 02:45:23.115235 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.931150 (+2815915us) service_pool.cc:227] Handling call
0806 02:45:25.931457 (+ 307us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260806 02:45:25.932147 21080 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:58094 (request call id 40) took 2759 ms. Trace:
I20260806 02:45:25.932251 21080 rpcz_store.cc:276] 0806 02:45:23.172211 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.931811 (+2759600us) service_pool.cc:227] Handling call
0806 02:45:25.932091 (+ 280us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260806 02:45:25.927476 21077 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:58094 (request call id 21) took 3359 ms. Trace:
I20260806 02:45:25.932790 21077 rpcz_store.cc:276] 0806 02:45:22.567621 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:22.706921 (+139300us) service_pool.cc:227] Handling call
0806 02:45:23.532395 (+825474us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0806 02:45:25.927003 (+2394608us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0806 02:45:25.927206 (+ 203us) catalog_manager.cc:3616] Apply alter schema
0806 02:45:25.927436 (+ 230us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":598439,"Hive Metastore.run_cpu_time_us":770,"Hive Metastore.run_wall_time_us":408712,"mutex_wait_us":17096}
I20260806 02:45:25.933575 21077 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:58094 (request call id 41) took 2118 ms. Trace:
I20260806 02:45:25.933786 21077 rpcz_store.cc:276] 0806 02:45:23.814845 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.933073 (+2118228us) service_pool.cc:227] Handling call
0806 02:45:25.933534 (+ 461us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260806 02:45:25.933960 22448 tablet_replica.cc:330] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:25.934074 21077 master_service.cc:946] ReplaceTablet: received request to replace tablet d8c3b721c7bd4c9b81ef7744c1230599 from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:25.934191 22448 raft_consensus.cc:2229] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:25.934478 21077 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:58094 (request call id 42) took 1968 ms. Trace:
I20260806 02:45:25.934551 22448 raft_consensus.cc:2258] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:25.934576 21077 rpcz_store.cc:276] 0806 02:45:23.966420 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.933972 (+1967552us) service_pool.cc:227] Handling call
0806 02:45:25.934411 (+ 439us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260806 02:45:25.935158 21077 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:58094 (request call id 43) took 1670 ms. Trace:
I20260806 02:45:25.935256 21077 rpcz_store.cc:276] 0806 02:45:24.264650 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.934803 (+1670153us) service_pool.cc:227] Handling call
0806 02:45:25.935102 (+ 299us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260806 02:45:25.935855 21077 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:58094 (request call id 44) took 1561 ms. Trace:
I20260806 02:45:25.936039 21077 rpcz_store.cc:276] 0806 02:45:24.374049 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.935405 (+1561356us) service_pool.cc:227] Handling call
0806 02:45:25.935703 (+ 298us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260806 02:45:25.936439 22448 ts_tablet_manager.cc:1905] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:25.936460 21077 master_service.cc:946] ReplaceTablet: received request to replace tablet d571f1c27e944cb4874a6c30a141c2ad from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:25.936865 21077 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:58094 (request call id 45) took 1310 ms. Trace:
I20260806 02:45:25.937047 21077 rpcz_store.cc:276] 0806 02:45:24.626375 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.936244 (+1309869us) service_pool.cc:227] Handling call
0806 02:45:25.936794 (+ 550us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260806 02:45:25.927948 21079 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:58094 (request call id 24) took 3251 ms. Trace:
I20260806 02:45:25.937456 21079 rpcz_store.cc:276] 0806 02:45:22.676067 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:22.930935 (+254868us) service_pool.cc:227] Handling call
0806 02:45:23.532510 (+601575us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0806 02:45:25.927879 (+2395369us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":1007012,"Hive Metastore.run_cpu_time_us":838,"Hive Metastore.run_wall_time_us":1103161}
I20260806 02:45:25.937896 21079 master_service.cc:946] ReplaceTablet: received request to replace tablet d8c3b721c7bd4c9b81ef7744c1230599 from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:25.938354 21079 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:58094 (request call id 47) took 1306 ms. Trace:
I20260806 02:45:25.938484 21079 rpcz_store.cc:276] 0806 02:45:24.631593 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.937777 (+1306184us) service_pool.cc:227] Handling call
0806 02:45:25.938321 (+ 544us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260806 02:45:25.938714 21079 master_service.cc:946] ReplaceTablet: received request to replace tablet d8c3b721c7bd4c9b81ef7744c1230599 from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:25.939079 21079 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:58094 (request call id 48) took 1307 ms. Trace:
I20260806 02:45:25.939168 21079 rpcz_store.cc:276] 0806 02:45:24.631768 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.938656 (+1306888us) service_pool.cc:227] Handling call
0806 02:45:25.939019 (+ 363us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260806 02:45:25.940856 21079 rpcz_store.cc:275] Call kudu.master.MasterService.GetMasterRegistration from 127.0.0.1:58090 (request call id 15) took 2379 ms. Trace:
I20260806 02:45:25.941056 21079 rpcz_store.cc:276] 0806 02:45:23.561485 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.940288 (+2378803us) service_pool.cc:227] Handling call
0806 02:45:25.940685 (+ 397us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260806 02:45:25.958137 21273 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46462:
name: "default.Table_c453f255605e4419859a21f098c7243e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:25.958205 21270 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46462:
name: "default.Table_0febb4673dff4e20ae8124efba9627c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:25.959172 21269 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46462:
name: "default.Table_d7249617a95049389b90db3bd551ace4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:25.962473 22407 consensus_queue.cc:1035] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:25.966440 21079 rpcz_store.cc:275] Call kudu.master.MasterService.GetMasterRegistration from 127.0.0.1:58058 (request call id 2) took 2155 ms. Trace:
I20260806 02:45:25.966667 21079 rpcz_store.cc:276] 0806 02:45:23.811088 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.965767 (+2154679us) service_pool.cc:227] Handling call
0806 02:45:25.966358 (+ 591us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260806 02:45:25.970396 21077 master_service.cc:946] ReplaceTablet: received request to replace tablet d571f1c27e944cb4874a6c30a141c2ad from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:25.971150 21077 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:58094 (request call id 46) took 1339 ms. Trace:
I20260806 02:45:25.971349 21077 rpcz_store.cc:276] 0806 02:45:24.631503 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.937277 (+1305774us) service_pool.cc:227] Handling call
0806 02:45:25.971010 (+ 33733us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260806 02:45:25.974431 21977 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.20.60.60:42309)
I20260806 02:45:25.980947 21268 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46462:
table { table_name: "default.Table_e24bd493305d46a4977ed544ce47cc83" } new_table_name: "default.Table_8e31b0866b344f85b9631a2e31b68af1" modify_external_catalogs: true
I20260806 02:45:25.981993 21266 master_service.cc:946] ReplaceTablet: received request to replace tablet d8c3b721c7bd4c9b81ef7744c1230599 from {username='slave'} at 127.0.0.1:46462
I20260806 02:45:25.988404 21264 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46462:
name: "default.Table_c5be13d5ad2c4a479d2cc9074d14038f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:25.990237 21263 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46462:
name: "default.Table_f243c93fbf57419bb6c685c08775e6ba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:25.995124 21083 rpcz_store.cc:276] 0806 02:45:22.971098 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.921948 (+2950850us) service_pool.cc:227] Handling call
0806 02:45:25.922329 (+ 381us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260806 02:45:26.001211 21266 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Sending DeleteTablet for 3 replicas of tablet d8c3b721c7bd4c9b81ef7744c1230599
I20260806 02:45:26.001897 21266 catalog_manager.cc:6565] ReplaceTablet: tablet d8c3b721c7bd4c9b81ef7744c1230599 deleted and replaced by tablet 0c861ce49ff4402eb2ef0b6430f2b8e8
I20260806 02:45:26.002200 22448 ts_tablet_manager.cc:1918] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:26.002449 22448 log.cc:1196] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/d571f1c27e944cb4874a6c30a141c2ad
I20260806 02:45:26.002559 21681 tablet_service.cc:1514] Processing DeleteTablet for tablet d8c3b721c7bd4c9b81ef7744c1230599 with delete_type TABLET_DATA_DELETED (replaced by tablet 0c861ce49ff4402eb2ef0b6430f2b8e8) from {username='slave'} at 127.0.0.1:38406
I20260806 02:45:26.002698 21266 master_service.cc:946] ReplaceTablet: received request to replace tablet d571f1c27e944cb4874a6c30a141c2ad from {username='slave'} at 127.0.0.1:46462
I20260806 02:45:26.002872 22444 tablet_replica.cc:330] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:26.003108 22444 raft_consensus.cc:2229] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:26.003314 22444 raft_consensus.cc:2258] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:26.003444 22448 ts_tablet_manager.cc:1939] T d571f1c27e944cb4874a6c30a141c2ad P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:26.003585 21266 master_service.cc:946] ReplaceTablet: received request to replace tablet d8c3b721c7bd4c9b81ef7744c1230599 from {username='slave'} at 127.0.0.1:46462
I20260806 02:45:26.003985 22444 ts_tablet_manager.cc:1905] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:26.004127 21266 master_service.cc:946] ReplaceTablet: received request to replace tablet d8c3b721c7bd4c9b81ef7744c1230599 from {username='slave'} at 127.0.0.1:46462
I20260806 02:45:26.004787 21266 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46462:
name: "default.Table_5cea2d6ea74c4c2f9366162462afcd71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:26.005514 21495 tablet_service.cc:1514] Processing DeleteTablet for tablet d8c3b721c7bd4c9b81ef7744c1230599 with delete_type TABLET_DATA_DELETED (replaced by tablet 0c861ce49ff4402eb2ef0b6430f2b8e8) from {username='slave'} at 127.0.0.1:33118
I20260806 02:45:26.005910 21863 tablet_service.cc:1514] Processing DeleteTablet for tablet d8c3b721c7bd4c9b81ef7744c1230599 with delete_type TABLET_DATA_DELETED (replaced by tablet 0c861ce49ff4402eb2ef0b6430f2b8e8) from {username='slave'} at 127.0.0.1:56374
I20260806 02:45:26.006312 21251 catalog_manager.cc:4841] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): tablet d571f1c27e944cb4874a6c30a141c2ad (table default.table_c94f1c901de74f71bc21408cd63fa3ea [id=2981a18bee6a4705ae5547688e2728f8]) successfully deleted
I20260806 02:45:26.006989 22445 tablet_replica.cc:330] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:26.007310 22445 raft_consensus.cc:2229] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:45:26.007755 22445 raft_consensus.cc:2258] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:26.008613 22445 ts_tablet_manager.cc:1905] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:26.013464 22448 tablet_replica.cc:330] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:26.013721 22448 raft_consensus.cc:2229] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:26.013927 22448 raft_consensus.cc:2258] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:26.014642 22448 ts_tablet_manager.cc:1905] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:26.018639 22445 ts_tablet_manager.cc:1918] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:26.023517 21495 tablet_service.cc:1467] Processing CreateTablet for tablet 0c861ce49ff4402eb2ef0b6430f2b8e8 (DEFAULT_TABLE table=default.table_f36563aa9b94423ebebd9741529ba14c [id=f9ea378b10f94d99a856cf000c426373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.024055 21495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c861ce49ff4402eb2ef0b6430f2b8e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.025738 21681 tablet_service.cc:1467] Processing CreateTablet for tablet 0c861ce49ff4402eb2ef0b6430f2b8e8 (DEFAULT_TABLE table=default.table_f36563aa9b94423ebebd9741529ba14c [id=f9ea378b10f94d99a856cf000c426373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.026333 21681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c861ce49ff4402eb2ef0b6430f2b8e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.032460 22444 ts_tablet_manager.cc:1918] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:26.034107 22407 consensus_queue.cc:1035] T 3196f6ee70d34d298b7db56d9759fab7 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:26.037139 22444 log.cc:1196] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/d8c3b721c7bd4c9b81ef7744c1230599
I20260806 02:45:26.038061 22444 ts_tablet_manager.cc:1939] T d8c3b721c7bd4c9b81ef7744c1230599 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:26.022776 21863 tablet_service.cc:1467] Processing CreateTablet for tablet 0c861ce49ff4402eb2ef0b6430f2b8e8 (DEFAULT_TABLE table=default.table_f36563aa9b94423ebebd9741529ba14c [id=f9ea378b10f94d99a856cf000c426373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.045389 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c861ce49ff4402eb2ef0b6430f2b8e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.047291 21255 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet d8c3b721c7bd4c9b81ef7744c1230599 (table default.table_f36563aa9b94423ebebd9741529ba14c [id=f9ea378b10f94d99a856cf000c426373]) successfully deleted
I20260806 02:45:26.053555 22445 log.cc:1196] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/d8c3b721c7bd4c9b81ef7744c1230599
I20260806 02:45:26.055768 22459 tablet_bootstrap.cc:492] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:26.057168 22445 ts_tablet_manager.cc:1939] T d8c3b721c7bd4c9b81ef7744c1230599 P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:26.058807 21251 catalog_manager.cc:4841] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): tablet d8c3b721c7bd4c9b81ef7744c1230599 (table default.table_f36563aa9b94423ebebd9741529ba14c [id=f9ea378b10f94d99a856cf000c426373]) successfully deleted
I20260806 02:45:26.062786 22463 tablet_bootstrap.cc:492] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:26.079834 22463 tablet_bootstrap.cc:654] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.098363 22463 tablet_bootstrap.cc:492] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:26.099416 22463 ts_tablet_manager.cc:1397] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.037s user 0.009s sys 0.000s
I20260806 02:45:26.101675 22464 tablet_bootstrap.cc:492] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:26.104472 22464 tablet_bootstrap.cc:654] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.106812 22428 tablet.cc:1722] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:26.105785 22427 tablet.cc:1722] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:26.125275 22459 tablet_bootstrap.cc:654] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.129182 22448 ts_tablet_manager.cc:1918] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:26.129488 22448 log.cc:1196] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/d8c3b721c7bd4c9b81ef7744c1230599
I20260806 02:45:26.130024 22448 ts_tablet_manager.cc:1939] T d8c3b721c7bd4c9b81ef7744c1230599 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:26.131850 21251 catalog_manager.cc:4841] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): tablet d8c3b721c7bd4c9b81ef7744c1230599 (table default.table_f36563aa9b94423ebebd9741529ba14c [id=f9ea378b10f94d99a856cf000c426373]) successfully deleted
I20260806 02:45:26.133499 22426 tablet.cc:1722] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:26.144138 22463 raft_consensus.cc:348] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:26.144678 22463 raft_consensus.cc:374] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.145051 22463 raft_consensus.cc:729] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.145975 22463 consensus_queue.cc:260] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:26.150270 22463 ts_tablet_manager.cc:1428] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.050s user 0.003s sys 0.000s
I20260806 02:45:26.165393 22459 tablet_bootstrap.cc:492] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:26.165592 22459 ts_tablet_manager.cc:1397] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.110s user 0.004s sys 0.006s
I20260806 02:45:26.166766 22459 raft_consensus.cc:348] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:26.167268 22459 raft_consensus.cc:374] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.167451 22459 raft_consensus.cc:729] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.168392 22459 consensus_queue.cc:260] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:26.170360 22459 ts_tablet_manager.cc:1428] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260806 02:45:26.179207 22464 tablet_bootstrap.cc:492] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:26.179571 22464 ts_tablet_manager.cc:1397] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.078s user 0.000s sys 0.006s
I20260806 02:45:26.181031 22464 raft_consensus.cc:348] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:26.181519 22464 raft_consensus.cc:374] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.181727 22464 raft_consensus.cc:729] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.182171 22464 consensus_queue.cc:260] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:26.187494 22464 ts_tablet_manager.cc:1428] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260806 02:45:26.263648 21271 catalog_manager.cc:2685] deleted HMS catalog entry for table default.table_f36563aa9b94423ebebd9741529ba14c [id=f9ea378b10f94d99a856cf000c426373]
W20260806 02:45:26.267657 21272 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_e8df6dde49954df7b57567fc0cd12872 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:45:26.273435 21345 catalog_manager.cc:2709] Deleting table default.table_f36563aa9b94423ebebd9741529ba14c [id=f9ea378b10f94d99a856cf000c426373] in response to Hive Metastore notification log event 21
I20260806 02:45:26.283388 21345 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Sending DeleteTablet for 3 replicas of tablet 0c861ce49ff4402eb2ef0b6430f2b8e8
I20260806 02:45:26.284243 21681 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c861ce49ff4402eb2ef0b6430f2b8e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:45:26 UTC) from {username='slave'} at 127.0.0.1:38406
I20260806 02:45:26.284559 22444 tablet_replica.cc:330] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
W20260806 02:45:26.284583 21273 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_c453f255605e4419859a21f098c7243e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:45:26.284849 22444 raft_consensus.cc:2229] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:26.285104 22444 raft_consensus.cc:2258] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Raft consensus is shut down!
W20260806 02:45:26.286036 21270 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_0febb4673dff4e20ae8124efba9627c8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:45:26.286186 22444 ts_tablet_manager.cc:1905] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 02:45:26.287245 21269 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d7249617a95049389b90db3bd551ace4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 02:45:26.288636 21264 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_c5be13d5ad2c4a479d2cc9074d14038f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 02:45:26.289794 21263 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f243c93fbf57419bb6c685c08775e6ba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 02:45:26.290903 21266 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_5cea2d6ea74c4c2f9366162462afcd71 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:45:26.291256 21076 catalog_manager.cc:2158] deleted orphaned HMS catalog entry for table [id=b4cb7469fede4d7cb3ae9bc653f5eec9]
I20260806 02:45:26.291782 21076 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:58094 (request call id 22) took 3684 ms. Trace:
I20260806 02:45:26.291916 21076 rpcz_store.cc:276] 0806 02:45:22.607566 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:22.888525 (+280959us) service_pool.cc:227] Handling call
0806 02:45:23.533159 (+644634us) catalog_manager.cc:2075] Acquired catalog manager lock
0806 02:45:23.533713 (+ 554us) catalog_manager.cc:2121] Created new table and tablet info
0806 02:45:25.683089 (+2149376us) catalog_manager.cc:2147] Created new table in HMS catalog
0806 02:45:25.684005 (+ 916us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:25.684009 (+ 4us) write_op.cc:454] Released schema lock
0806 02:45:25.684438 (+ 429us) catalog_manager.cc:2155] Rolling back HMS table creation
0806 02:45:26.291712 (+607274us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0806 02:45:25.683889 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica a225e002d607451a94b3686f10aeea9a is not leader of this config: current role FOLLOWER)
0806 02:45:25.683906 (+ 17us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:25.683913 (+ 7us) write_op.cc:454] Released schema lock
0806 02:45:25.683917 (+ 4us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":2307032,"Hive Metastore.run_cpu_time_us":604,"Hive Metastore.run_wall_time_us":448226,"child_traces":[["op",{}]]}
I20260806 02:45:26.295708 21271 master_service.cc:946] ReplaceTablet: received request to replace tablet d571f1c27e944cb4874a6c30a141c2ad from {username='slave'} at 127.0.0.1:46462
I20260806 02:45:26.298818 22444 ts_tablet_manager.cc:1918] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:26.299057 22444 log.cc:1196] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/0c861ce49ff4402eb2ef0b6430f2b8e8
I20260806 02:45:26.299546 22444 ts_tablet_manager.cc:1939] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:26.300786 21255 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet 0c861ce49ff4402eb2ef0b6430f2b8e8 (table default.table_f36563aa9b94423ebebd9741529ba14c [id=f9ea378b10f94d99a856cf000c426373]) successfully deleted
I20260806 02:45:26.303299 21495 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c861ce49ff4402eb2ef0b6430f2b8e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:45:26 UTC) from {username='slave'} at 127.0.0.1:33118
I20260806 02:45:26.304699 22448 tablet_replica.cc:330] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:26.304944 22448 raft_consensus.cc:2229] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:26.305227 22448 raft_consensus.cc:2258] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:26.305840 21863 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c861ce49ff4402eb2ef0b6430f2b8e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:45:26 UTC) from {username='slave'} at 127.0.0.1:56374
I20260806 02:45:26.306260 22445 tablet_replica.cc:330] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:26.306556 22445 raft_consensus.cc:2229] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:26.306869 22445 raft_consensus.cc:2258] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:26.311196 22448 ts_tablet_manager.cc:1905] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:26.313997 22445 ts_tablet_manager.cc:1905] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:26.325912 22448 ts_tablet_manager.cc:1918] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:26.331185 22445 ts_tablet_manager.cc:1918] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:26.331463 22448 log.cc:1196] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/0c861ce49ff4402eb2ef0b6430f2b8e8
I20260806 02:45:26.331974 22448 ts_tablet_manager.cc:1939] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:26.332690 22445 log.cc:1196] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/0c861ce49ff4402eb2ef0b6430f2b8e8
I20260806 02:45:26.333289 22445 ts_tablet_manager.cc:1939] T 0c861ce49ff4402eb2ef0b6430f2b8e8 P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:26.334571 21251 catalog_manager.cc:4841] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): tablet 0c861ce49ff4402eb2ef0b6430f2b8e8 (table default.table_f36563aa9b94423ebebd9741529ba14c [id=f9ea378b10f94d99a856cf000c426373]) successfully deleted
I20260806 02:45:26.346866 21271 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e] alter complete (version 1)
I20260806 02:45:26.347877 21495 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c861ce49ff4402eb2ef0b6430f2b8e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:45:26 UTC) from {username='slave'} at 127.0.0.1:33118
I20260806 02:45:26.363976 21251 catalog_manager.cc:4841] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): tablet 0c861ce49ff4402eb2ef0b6430f2b8e8 (table default.table_f36563aa9b94423ebebd9741529ba14c [id=f9ea378b10f94d99a856cf000c426373]) successfully deleted
W20260806 02:45:26.365170 21251 catalog_manager.cc:4808] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): delete failed for tablet 0c861ce49ff4402eb2ef0b6430f2b8e8 because the tablet was not found. No further retry: Not found: Tablet not found: 0c861ce49ff4402eb2ef0b6430f2b8e8
I20260806 02:45:26.371237 21271 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46462:
table { table_name: "default.Table_21221d53bf9f4c46a5ffe671a1ffe424" } new_table_name: "default.Table_fb2ecedbb20f49d2a659b25c01f24887" modify_external_catalogs: true
I20260806 02:45:26.383592 21073 catalog_manager.cc:2158] deleted orphaned HMS catalog entry for table [id=fe86c442832f49bc8efdb723bab0fc10]
I20260806 02:45:26.384044 21073 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:58094 (request call id 25) took 3678 ms. Trace:
I20260806 02:45:26.384225 21073 rpcz_store.cc:276] 0806 02:45:22.705193 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:22.972051 (+266858us) service_pool.cc:227] Handling call
0806 02:45:23.533240 (+561189us) catalog_manager.cc:2075] Acquired catalog manager lock
0806 02:45:23.534433 (+ 1193us) catalog_manager.cc:2121] Created new table and tablet info
0806 02:45:25.854663 (+2320230us) catalog_manager.cc:2147] Created new table in HMS catalog
0806 02:45:25.855646 (+ 983us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:25.855650 (+ 4us) write_op.cc:454] Released schema lock
0806 02:45:25.856074 (+ 424us) catalog_manager.cc:2155] Rolling back HMS table creation
0806 02:45:26.383954 (+527880us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0806 02:45:25.855558 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica a225e002d607451a94b3686f10aeea9a is not leader of this config: current role FOLLOWER)
0806 02:45:25.855580 (+ 22us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:25.855587 (+ 7us) write_op.cc:454] Released schema lock
0806 02:45:25.855591 (+ 4us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":2583058,"Hive Metastore.run_cpu_time_us":558,"Hive Metastore.run_wall_time_us":263932,"child_traces":[["op",{}]]}
I20260806 02:45:26.419862 21274 catalog_manager.cc:3411] renamed table default.table_8797f6f61c444bd0a4e4555f6ca2fe30 [id=370d5895c35f4c96878270e71cac2443] in HMS: new name default.table_97ee2606f8aa47dcb2c67a9ae29dd3fb
I20260806 02:45:26.430191 21082 catalog_manager.cc:2148] created HMS catalog entry for table [id=62dba0d42b05487a82b517a5393caa86]
W20260806 02:45:26.431344 21082 catalog_manager.cc:2176] Illegal state: an error occurred while writing to the sys-catalog: replica a225e002d607451a94b3686f10aeea9a is not leader of this config: current role FOLLOWER
I20260806 02:45:26.481230 21272 catalog_manager.cc:2148] created HMS catalog entry for table [id=a0cffb03ae7641f8848b8d2aab0fca8b]
I20260806 02:45:26.489696 21074 catalog_manager.cc:2685] deleted HMS catalog entry for table default.table_a7715d8e001f48b2899c3c2a56a18a06 [id=21d847eb419e4c0998a01f87c8cd92da]
I20260806 02:45:26.490679 21074 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:58094 (request call id 30) took 3682 ms. Trace:
I20260806 02:45:26.490859 21074 rpcz_store.cc:276] 0806 02:45:22.808212 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:24.196407 (+1388195us) service_pool.cc:227] Handling call
0806 02:45:24.217010 (+ 20603us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x615000033024. stack: 00007f1a1efb666e 00007f1a1efb6430 00007f1a1e03d5ed 00000000008ea064 00000000008ea02d 00007f1a2d6cf27d 00007f1a2d6cf373 00007f1a2a4f1361 00007f1a31e6466a 00007f1a31fed2e6 00007f1a292e0126 00007f1a292dfe55 00007f1a25af29be 00007f1a25af179e 00007f1a25af7b25 00007f1a25af854f
0806 02:45:25.926746 (+1709736us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0806 02:45:26.490565 (+563819us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":502979,"Hive Metastore.run_cpu_time_us":270,"Hive Metastore.run_wall_time_us":59597,"spinlock_wait_cycles":46432640}
I20260806 02:45:26.492746 21272 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46462:
table { table_name: "default.Table_00c06d2b7d6d40658a4390f4ee3bbab5" } new_table_name: "default.Table_6fff046fe11443558b808f8516187245" modify_external_catalogs: true
I20260806 02:45:26.509653 21681 tablet_service.cc:1467] Processing CreateTablet for tablet a6e1420c19fc44b3901f653f137ec6c1 (DEFAULT_TABLE table=default.table_e8df6dde49954df7b57567fc0cd12872 [id=a0cffb03ae7641f8848b8d2aab0fca8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.510211 21681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e1420c19fc44b3901f653f137ec6c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.514847 21495 tablet_service.cc:1467] Processing CreateTablet for tablet a6e1420c19fc44b3901f653f137ec6c1 (DEFAULT_TABLE table=default.table_e8df6dde49954df7b57567fc0cd12872 [id=a0cffb03ae7641f8848b8d2aab0fca8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.515443 21495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e1420c19fc44b3901f653f137ec6c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.518126 21863 tablet_service.cc:1467] Processing CreateTablet for tablet a6e1420c19fc44b3901f653f137ec6c1 (DEFAULT_TABLE table=default.table_e8df6dde49954df7b57567fc0cd12872 [id=a0cffb03ae7641f8848b8d2aab0fca8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.518673 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e1420c19fc44b3901f653f137ec6c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.531097 21273 catalog_manager.cc:2148] created HMS catalog entry for table [id=ad046cf7c81b42e5afcde9d116b3fc26]
I20260806 02:45:26.539415 21273 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46462:
table { table_name: "default.Table_ddb47e155c91465f8e38147105f2a6bd" } new_table_name: "default.Table_a7570be8f5c74682ac8df015be8417b6" modify_external_catalogs: true
I20260806 02:45:26.544118 21081 catalog_manager.cc:2685] deleted HMS catalog entry for table default.table_c2d5cb4cc8a44eaca15ba851ecc366c8 [id=ccf2db5226644b498abe88bba95d6fef]
I20260806 02:45:26.544854 21081 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:58094 (request call id 33) took 3686 ms. Trace:
I20260806 02:45:26.545004 21081 rpcz_store.cc:276] 0806 02:45:22.858675 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:24.694765 (+1836090us) service_pool.cc:227] Handling call
0806 02:45:25.927678 (+1232913us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0806 02:45:26.544807 (+617129us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":561730,"Hive Metastore.run_cpu_time_us":196,"Hive Metastore.run_wall_time_us":54285}
I20260806 02:45:26.553304 21682 tablet_service.cc:1467] Processing CreateTablet for tablet 81dd98ae43b7429992d48eeaa9f17a70 (DEFAULT_TABLE table=default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.553921 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81dd98ae43b7429992d48eeaa9f17a70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.564263 21494 tablet_service.cc:1467] Processing CreateTablet for tablet 81dd98ae43b7429992d48eeaa9f17a70 (DEFAULT_TABLE table=default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.564910 21494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81dd98ae43b7429992d48eeaa9f17a70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.570242 21865 tablet_service.cc:1467] Processing CreateTablet for tablet 81dd98ae43b7429992d48eeaa9f17a70 (DEFAULT_TABLE table=default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.570688 21865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81dd98ae43b7429992d48eeaa9f17a70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.577812 22464 tablet_bootstrap.cc:492] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:26.579969 22464 tablet_bootstrap.cc:654] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.585273 22459 tablet_bootstrap.cc:492] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:26.587339 22459 tablet_bootstrap.cc:654] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.589854 22463 tablet_bootstrap.cc:492] T a6e1420c19fc44b3901f653f137ec6c1 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:26.592185 22463 tablet_bootstrap.cc:654] T a6e1420c19fc44b3901f653f137ec6c1 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.597234 22463 tablet_bootstrap.cc:492] T a6e1420c19fc44b3901f653f137ec6c1 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:26.597573 22463 ts_tablet_manager.cc:1397] T a6e1420c19fc44b3901f653f137ec6c1 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260806 02:45:26.598824 22463 raft_consensus.cc:348] T a6e1420c19fc44b3901f653f137ec6c1 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.599303 22463 raft_consensus.cc:374] T a6e1420c19fc44b3901f653f137ec6c1 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.599488 22463 raft_consensus.cc:729] T a6e1420c19fc44b3901f653f137ec6c1 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.599859 21270 catalog_manager.cc:2148] created HMS catalog entry for table [id=5cb271f99441401cad4fd2e40e943a6e]
I20260806 02:45:26.601827 22459 tablet_bootstrap.cc:492] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:26.602018 22459 ts_tablet_manager.cc:1397] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.005s
I20260806 02:45:26.603404 22463 consensus_queue.cc:260] T a6e1420c19fc44b3901f653f137ec6c1 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.602828 22459 raft_consensus.cc:348] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.606122 22459 raft_consensus.cc:374] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.606302 22459 raft_consensus.cc:729] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.606647 22459 consensus_queue.cc:260] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.614921 22464 tablet_bootstrap.cc:492] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:26.615128 22464 ts_tablet_manager.cc:1397] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.037s user 0.005s sys 0.007s
I20260806 02:45:26.616011 22464 raft_consensus.cc:348] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.616281 22464 raft_consensus.cc:374] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.616369 22464 raft_consensus.cc:729] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.616685 22464 consensus_queue.cc:260] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.617827 22464 ts_tablet_manager.cc:1428] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 02:45:26.618245 22464 tablet_bootstrap.cc:492] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:26.619371 22459 ts_tablet_manager.cc:1428] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.017s user 0.003s sys 0.000s
I20260806 02:45:26.620455 22464 tablet_bootstrap.cc:654] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.623673 22459 tablet_bootstrap.cc:492] T a6e1420c19fc44b3901f653f137ec6c1 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:26.626570 22463 ts_tablet_manager.cc:1428] T a6e1420c19fc44b3901f653f137ec6c1 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.029s user 0.002s sys 0.000s
I20260806 02:45:26.630206 22459 tablet_bootstrap.cc:654] T a6e1420c19fc44b3901f653f137ec6c1 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.631980 21494 tablet_service.cc:1467] Processing CreateTablet for tablet ba32fbee948d4adf96164d36673a1db6 (DEFAULT_TABLE table=default.table_0febb4673dff4e20ae8124efba9627c8 [id=5cb271f99441401cad4fd2e40e943a6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.632066 21863 tablet_service.cc:1467] Processing CreateTablet for tablet ba32fbee948d4adf96164d36673a1db6 (DEFAULT_TABLE table=default.table_0febb4673dff4e20ae8124efba9627c8 [id=5cb271f99441401cad4fd2e40e943a6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.632449 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba32fbee948d4adf96164d36673a1db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.632520 21494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba32fbee948d4adf96164d36673a1db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.637933 21682 tablet_service.cc:1467] Processing CreateTablet for tablet ba32fbee948d4adf96164d36673a1db6 (DEFAULT_TABLE table=default.table_0febb4673dff4e20ae8124efba9627c8 [id=5cb271f99441401cad4fd2e40e943a6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.640339 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba32fbee948d4adf96164d36673a1db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.651855 22463 tablet_bootstrap.cc:492] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:26.653623 22459 tablet_bootstrap.cc:492] T a6e1420c19fc44b3901f653f137ec6c1 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:26.653798 22459 ts_tablet_manager.cc:1397] T a6e1420c19fc44b3901f653f137ec6c1 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.030s user 0.009s sys 0.015s
I20260806 02:45:26.654879 22459 raft_consensus.cc:348] T a6e1420c19fc44b3901f653f137ec6c1 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.651134 21270 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46462:
table { table_name: "default.Table_ea1a38afd92a4b15bfabb80e58f8c644" } modify_external_catalogs: true
I20260806 02:45:26.655200 22459 raft_consensus.cc:374] T a6e1420c19fc44b3901f653f137ec6c1 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.655447 22459 raft_consensus.cc:729] T a6e1420c19fc44b3901f653f137ec6c1 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.655714 22459 consensus_queue.cc:260] T a6e1420c19fc44b3901f653f137ec6c1 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.655500 21270 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46462:
table { table_name: "default.Table_ea1a38afd92a4b15bfabb80e58f8c644" } modify_external_catalogs: true
I20260806 02:45:26.659773 22459 ts_tablet_manager.cc:1428] T a6e1420c19fc44b3901f653f137ec6c1 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260806 02:45:26.660742 22463 tablet_bootstrap.cc:654] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.662416 22459 tablet_bootstrap.cc:492] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:26.663188 21082 catalog_manager.cc:2158] deleted orphaned HMS catalog entry for table [id=62dba0d42b05487a82b517a5393caa86]
I20260806 02:45:26.663623 21082 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:58094 (request call id 27) took 3896 ms. Trace:
I20260806 02:45:26.663756 21082 rpcz_store.cc:276] 0806 02:45:22.766931 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:23.716845 (+949914us) service_pool.cc:227] Handling call
0806 02:45:25.909480 (+2192635us) catalog_manager.cc:2075] Acquired catalog manager lock
0806 02:45:25.910010 (+ 530us) catalog_manager.cc:2121] Created new table and tablet info
0806 02:45:26.430149 (+520139us) catalog_manager.cc:2147] Created new table in HMS catalog
0806 02:45:26.431128 (+ 979us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:26.431132 (+ 4us) write_op.cc:454] Released schema lock
0806 02:45:26.431468 (+ 336us) catalog_manager.cc:2155] Rolling back HMS table creation
0806 02:45:26.663554 (+232086us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0806 02:45:26.431024 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica a225e002d607451a94b3686f10aeea9a is not leader of this config: current role FOLLOWER)
0806 02:45:26.431051 (+ 27us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:26.431059 (+ 8us) write_op.cc:454] Released schema lock
0806 02:45:26.431064 (+ 5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":585493,"Hive Metastore.run_cpu_time_us":585,"Hive Metastore.run_wall_time_us":165648,"child_traces":[["op",{}]]}
I20260806 02:45:26.664769 22459 tablet_bootstrap.cc:654] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.670356 22463 tablet_bootstrap.cc:492] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:26.670624 22463 ts_tablet_manager.cc:1397] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.003s
I20260806 02:45:26.671656 22459 tablet_bootstrap.cc:492] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:26.671924 22459 ts_tablet_manager.cc:1397] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.001s
I20260806 02:45:26.672093 22463 raft_consensus.cc:348] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.672574 22463 raft_consensus.cc:374] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.672871 22463 raft_consensus.cc:729] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.673051 22459 raft_consensus.cc:348] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.673287 22463 consensus_queue.cc:260] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.673468 22459 raft_consensus.cc:374] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.673626 22459 raft_consensus.cc:729] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.673925 22459 consensus_queue.cc:260] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.674170 22464 tablet_bootstrap.cc:492] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:26.674327 22464 ts_tablet_manager.cc:1397] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.056s user 0.012s sys 0.001s
I20260806 02:45:26.674569 22459 ts_tablet_manager.cc:1428] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 02:45:26.679951 22463 ts_tablet_manager.cc:1428] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260806 02:45:26.680545 22463 tablet_bootstrap.cc:492] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:26.681948 22464 raft_consensus.cc:348] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.682286 22464 raft_consensus.cc:374] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.682495 22464 raft_consensus.cc:729] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.682998 22464 consensus_queue.cc:260] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.683405 22463 tablet_bootstrap.cc:654] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.685875 21269 catalog_manager.cc:2148] created HMS catalog entry for table [id=4179f72ea45043e39ddce8e93c3429e9]
I20260806 02:45:26.692755 22464 ts_tablet_manager.cc:1428] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.018s user 0.002s sys 0.001s
I20260806 02:45:26.696780 21269 master_service.cc:946] ReplaceTablet: received request to replace tablet b4e3f2c8eb2c4ea8a1b041be073bdd08 from {username='slave'} at 127.0.0.1:46462
I20260806 02:45:26.707234 21494 tablet_service.cc:1467] Processing CreateTablet for tablet f9fd42a920b24117a4d687fec13ad17b (DEFAULT_TABLE table=default.table_d7249617a95049389b90db3bd551ace4 [id=4179f72ea45043e39ddce8e93c3429e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.707691 21682 tablet_service.cc:1467] Processing CreateTablet for tablet f9fd42a920b24117a4d687fec13ad17b (DEFAULT_TABLE table=default.table_d7249617a95049389b90db3bd551ace4 [id=4179f72ea45043e39ddce8e93c3429e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.707839 21494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9fd42a920b24117a4d687fec13ad17b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.708186 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9fd42a920b24117a4d687fec13ad17b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.714319 21863 tablet_service.cc:1467] Processing CreateTablet for tablet f9fd42a920b24117a4d687fec13ad17b (DEFAULT_TABLE table=default.table_d7249617a95049389b90db3bd551ace4 [id=4179f72ea45043e39ddce8e93c3429e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.714922 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9fd42a920b24117a4d687fec13ad17b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.715485 22464 tablet_bootstrap.cc:492] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:26.719918 21269 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Sending DeleteTablet for 3 replicas of tablet b4e3f2c8eb2c4ea8a1b041be073bdd08
I20260806 02:45:26.721976 22464 tablet_bootstrap.cc:654] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.726132 22459 tablet_bootstrap.cc:492] T f9fd42a920b24117a4d687fec13ad17b P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:26.728525 21682 tablet_service.cc:1514] Processing DeleteTablet for tablet b4e3f2c8eb2c4ea8a1b041be073bdd08 with delete_type TABLET_DATA_DELETED (replaced by tablet 62b1e9f4216845739fc757c975eb8c02) from {username='slave'} at 127.0.0.1:38406
I20260806 02:45:26.729043 22444 tablet_replica.cc:330] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:26.729240 22459 tablet_bootstrap.cc:654] T f9fd42a920b24117a4d687fec13ad17b P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.729313 22444 raft_consensus.cc:2229] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:26.729646 21269 catalog_manager.cc:6565] ReplaceTablet: tablet b4e3f2c8eb2c4ea8a1b041be073bdd08 deleted and replaced by tablet 62b1e9f4216845739fc757c975eb8c02
I20260806 02:45:26.729696 22444 raft_consensus.cc:2258] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:26.729719 21495 tablet_service.cc:1514] Processing DeleteTablet for tablet b4e3f2c8eb2c4ea8a1b041be073bdd08 with delete_type TABLET_DATA_DELETED (replaced by tablet 62b1e9f4216845739fc757c975eb8c02) from {username='slave'} at 127.0.0.1:33118
I20260806 02:45:26.730072 22448 tablet_replica.cc:330] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:26.730319 22448 raft_consensus.cc:2229] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:26.730695 22448 raft_consensus.cc:2258] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:26.732005 22448 ts_tablet_manager.cc:1905] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:26.735674 22444 ts_tablet_manager.cc:1905] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:26.737694 21863 tablet_service.cc:1514] Processing DeleteTablet for tablet b4e3f2c8eb2c4ea8a1b041be073bdd08 with delete_type TABLET_DATA_DELETED (replaced by tablet 62b1e9f4216845739fc757c975eb8c02) from {username='slave'} at 127.0.0.1:56374
I20260806 02:45:26.738216 22445 tablet_replica.cc:330] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:26.738582 22445 raft_consensus.cc:2229] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:45:26.739086 22445 raft_consensus.cc:2258] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:26.740418 22445 ts_tablet_manager.cc:1905] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:26.746276 21681 tablet_service.cc:1467] Processing CreateTablet for tablet 62b1e9f4216845739fc757c975eb8c02 (DEFAULT_TABLE table=default.table_8797f6f61c444bd0a4e4555f6ca2fe30 [id=370d5895c35f4c96878270e71cac2443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.746794 21681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62b1e9f4216845739fc757c975eb8c02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.748286 22444 ts_tablet_manager.cc:1918] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:45:26.748602 22444 log.cc:1196] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/b4e3f2c8eb2c4ea8a1b041be073bdd08
I20260806 02:45:26.749171 22444 ts_tablet_manager.cc:1939] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:26.749703 22412 raft_consensus.cc:482] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:26.749951 22412 raft_consensus.cc:504] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.751484 21705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6e1420c19fc44b3901f653f137ec6c1" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" is_pre_election: true
I20260806 02:45:26.751981 21255 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet b4e3f2c8eb2c4ea8a1b041be073bdd08 (table default.table_8797f6f61c444bd0a4e4555f6ca2fe30 [id=370d5895c35f4c96878270e71cac2443]) successfully deleted
I20260806 02:45:26.752425 22448 ts_tablet_manager.cc:1918] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:45:26.752638 22448 log.cc:1196] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/b4e3f2c8eb2c4ea8a1b041be073bdd08
I20260806 02:45:26.753170 22448 ts_tablet_manager.cc:1939] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:26.758354 22412 leader_election.cc:288] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:26.759280 21520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6e1420c19fc44b3901f653f137ec6c1" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:26.759641 21520 raft_consensus.cc:2454] T a6e1420c19fc44b3901f653f137ec6c1 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:26.760311 21803 leader_election.cc:302] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:26.761379 21251 catalog_manager.cc:4841] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): tablet b4e3f2c8eb2c4ea8a1b041be073bdd08 (table default.table_8797f6f61c444bd0a4e4555f6ca2fe30 [id=370d5895c35f4c96878270e71cac2443]) successfully deleted
I20260806 02:45:26.762022 21494 tablet_service.cc:1467] Processing CreateTablet for tablet 62b1e9f4216845739fc757c975eb8c02 (DEFAULT_TABLE table=default.table_8797f6f61c444bd0a4e4555f6ca2fe30 [id=370d5895c35f4c96878270e71cac2443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.762549 21494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62b1e9f4216845739fc757c975eb8c02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.763266 21269 master_service.cc:946] ReplaceTablet: received request to replace tablet 3f0c5aff461940d999d0ed14292db320 from {username='slave'} at 127.0.0.1:46462
I20260806 02:45:26.764994 22412 raft_consensus.cc:2781] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:26.765218 22412 raft_consensus.cc:482] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:26.765165 21705 raft_consensus.cc:2454] T a6e1420c19fc44b3901f653f137ec6c1 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:26.765443 22412 raft_consensus.cc:3037] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:26.765952 21863 tablet_service.cc:1467] Processing CreateTablet for tablet 62b1e9f4216845739fc757c975eb8c02 (DEFAULT_TABLE table=default.table_8797f6f61c444bd0a4e4555f6ca2fe30 [id=370d5895c35f4c96878270e71cac2443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.766513 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62b1e9f4216845739fc757c975eb8c02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.770637 22412 raft_consensus.cc:504] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.772397 21520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6e1420c19fc44b3901f653f137ec6c1" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:26.772702 21520 raft_consensus.cc:3037] T a6e1420c19fc44b3901f653f137ec6c1 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:26.773571 21705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6e1420c19fc44b3901f653f137ec6c1" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
I20260806 02:45:26.777508 21705 raft_consensus.cc:3037] T a6e1420c19fc44b3901f653f137ec6c1 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:26.777705 21520 raft_consensus.cc:2454] T a6e1420c19fc44b3901f653f137ec6c1 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:26.778276 21803 leader_election.cc:302] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:26.779408 21269 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Sending DeleteTablet for 3 replicas of tablet 3f0c5aff461940d999d0ed14292db320
I20260806 02:45:26.780194 21495 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f0c5aff461940d999d0ed14292db320 with delete_type TABLET_DATA_DELETED (replaced by tablet aac61d5d826042e4bcebdcfde8b71a1e) from {username='slave'} at 127.0.0.1:33118
I20260806 02:45:26.780597 22448 tablet_replica.cc:330] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:26.780857 22448 raft_consensus.cc:2229] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:45:26.781286 22448 raft_consensus.cc:2258] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:26.781456 21705 raft_consensus.cc:2454] T a6e1420c19fc44b3901f653f137ec6c1 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:26.781584 22412 leader_election.cc:288] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Requested vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:26.781822 22412 raft_consensus.cc:2781] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:26.782027 22412 raft_consensus.cc:686] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [term 1 LEADER]: Becoming Leader. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Running, Role: LEADER
I20260806 02:45:26.782022 22448 ts_tablet_manager.cc:1905] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:26.783170 22463 tablet_bootstrap.cc:492] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:26.783380 22463 ts_tablet_manager.cc:1397] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.103s user 0.008s sys 0.030s
I20260806 02:45:26.783336 22412 consensus_queue.cc:237] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.784612 22463 raft_consensus.cc:348] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.784885 22463 raft_consensus.cc:374] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.785019 22463 raft_consensus.cc:729] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.785421 22463 consensus_queue.cc:260] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.785991 22463 ts_tablet_manager.cc:1428] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 02:45:26.787814 21865 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f0c5aff461940d999d0ed14292db320 with delete_type TABLET_DATA_DELETED (replaced by tablet aac61d5d826042e4bcebdcfde8b71a1e) from {username='slave'} at 127.0.0.1:56374
I20260806 02:45:26.790167 21269 catalog_manager.cc:6565] ReplaceTablet: tablet 3f0c5aff461940d999d0ed14292db320 deleted and replaced by tablet aac61d5d826042e4bcebdcfde8b71a1e
I20260806 02:45:26.789495 22463 tablet_bootstrap.cc:492] T f9fd42a920b24117a4d687fec13ad17b P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:26.791724 21682 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f0c5aff461940d999d0ed14292db320 with delete_type TABLET_DATA_DELETED (replaced by tablet aac61d5d826042e4bcebdcfde8b71a1e) from {username='slave'} at 127.0.0.1:38406
I20260806 02:45:26.792111 22444 tablet_replica.cc:330] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:26.792371 22444 raft_consensus.cc:2229] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:26.792632 22444 raft_consensus.cc:2258] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:26.794304 22444 ts_tablet_manager.cc:1905] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:26.796769 22448 ts_tablet_manager.cc:1918] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:26.796923 22448 log.cc:1196] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/3f0c5aff461940d999d0ed14292db320
I20260806 02:45:26.802774 22445 ts_tablet_manager.cc:1918] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:45:26.804554 22463 tablet_bootstrap.cc:654] T f9fd42a920b24117a4d687fec13ad17b P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.805903 22445 log.cc:1196] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/b4e3f2c8eb2c4ea8a1b041be073bdd08
I20260806 02:45:26.806882 22445 ts_tablet_manager.cc:1939] T b4e3f2c8eb2c4ea8a1b041be073bdd08 P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:26.809571 22444 ts_tablet_manager.cc:1918] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:26.809747 22444 log.cc:1196] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/3f0c5aff461940d999d0ed14292db320
I20260806 02:45:26.810268 22444 ts_tablet_manager.cc:1939] T 3f0c5aff461940d999d0ed14292db320 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:26.811386 21255 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet 3f0c5aff461940d999d0ed14292db320 (table default.table_a7715d8e001f48b2899c3c2a56a18a06 [id=21d847eb419e4c0998a01f87c8cd92da]) successfully deleted
I20260806 02:45:26.813822 21863 tablet_service.cc:1467] Processing CreateTablet for tablet aac61d5d826042e4bcebdcfde8b71a1e (DEFAULT_TABLE table=default.table_a7715d8e001f48b2899c3c2a56a18a06 [id=21d847eb419e4c0998a01f87c8cd92da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.814080 21494 tablet_service.cc:1467] Processing CreateTablet for tablet aac61d5d826042e4bcebdcfde8b71a1e (DEFAULT_TABLE table=default.table_a7715d8e001f48b2899c3c2a56a18a06 [id=21d847eb419e4c0998a01f87c8cd92da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.814440 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aac61d5d826042e4bcebdcfde8b71a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.814756 21494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aac61d5d826042e4bcebdcfde8b71a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.815129 21251 catalog_manager.cc:4841] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): tablet b4e3f2c8eb2c4ea8a1b041be073bdd08 (table default.table_8797f6f61c444bd0a4e4555f6ca2fe30 [id=370d5895c35f4c96878270e71cac2443]) successfully deleted
I20260806 02:45:26.816072 22445 tablet_replica.cc:330] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:26.816450 22445 raft_consensus.cc:2229] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:26.816776 22459 tablet_bootstrap.cc:492] T f9fd42a920b24117a4d687fec13ad17b P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:26.816859 22445 raft_consensus.cc:2258] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:26.817081 22459 ts_tablet_manager.cc:1397] T f9fd42a920b24117a4d687fec13ad17b P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.091s user 0.014s sys 0.018s
I20260806 02:45:26.817673 21681 tablet_service.cc:1467] Processing CreateTablet for tablet aac61d5d826042e4bcebdcfde8b71a1e (DEFAULT_TABLE table=default.table_a7715d8e001f48b2899c3c2a56a18a06 [id=21d847eb419e4c0998a01f87c8cd92da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.818179 21681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aac61d5d826042e4bcebdcfde8b71a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.818173 22459 raft_consensus.cc:348] T f9fd42a920b24117a4d687fec13ad17b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.818569 22459 raft_consensus.cc:374] T f9fd42a920b24117a4d687fec13ad17b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.818722 22459 raft_consensus.cc:729] T f9fd42a920b24117a4d687fec13ad17b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.819000 22445 ts_tablet_manager.cc:1905] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:26.819072 22459 consensus_queue.cc:260] T f9fd42a920b24117a4d687fec13ad17b P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.822588 22459 ts_tablet_manager.cc:1428] T f9fd42a920b24117a4d687fec13ad17b P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260806 02:45:26.823045 22459 tablet_bootstrap.cc:492] T 62b1e9f4216845739fc757c975eb8c02 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:26.823132 21269 catalog_manager.cc:5495] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3). New cstate: current_term: 1 leader_uuid: "8ebd96b903b944f3b06851324b737bd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: HEALTHY } } }
I20260806 02:45:26.828331 22459 tablet_bootstrap.cc:654] T 62b1e9f4216845739fc757c975eb8c02 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.829913 22448 ts_tablet_manager.cc:1939] T 3f0c5aff461940d999d0ed14292db320 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:26.831741 21251 catalog_manager.cc:4841] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): tablet 3f0c5aff461940d999d0ed14292db320 (table default.table_a7715d8e001f48b2899c3c2a56a18a06 [id=21d847eb419e4c0998a01f87c8cd92da]) successfully deleted
I20260806 02:45:26.832441 22463 tablet_bootstrap.cc:492] T f9fd42a920b24117a4d687fec13ad17b P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:26.832635 22463 ts_tablet_manager.cc:1397] T f9fd42a920b24117a4d687fec13ad17b P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.046s user 0.002s sys 0.002s
I20260806 02:45:26.833439 22463 raft_consensus.cc:348] T f9fd42a920b24117a4d687fec13ad17b P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.833806 22463 raft_consensus.cc:374] T f9fd42a920b24117a4d687fec13ad17b P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.833921 22463 raft_consensus.cc:729] T f9fd42a920b24117a4d687fec13ad17b P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.834187 22463 consensus_queue.cc:260] T f9fd42a920b24117a4d687fec13ad17b P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.835999 22463 ts_tablet_manager.cc:1428] T f9fd42a920b24117a4d687fec13ad17b P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 02:45:26.836310 22463 tablet_bootstrap.cc:492] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:26.853838 22463 tablet_bootstrap.cc:654] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.856019 22464 tablet_bootstrap.cc:492] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:26.856268 22464 ts_tablet_manager.cc:1397] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.141s user 0.022s sys 0.023s
I20260806 02:45:26.857048 21269 master_service.cc:946] ReplaceTablet: received request to replace tablet 3b3a4c79185241a7a8522135f0a513b3 from {username='slave'} at 127.0.0.1:46462
I20260806 02:45:26.857364 22464 raft_consensus.cc:348] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.857649 22464 raft_consensus.cc:374] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.857745 22464 raft_consensus.cc:729] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.858184 22464 consensus_queue.cc:260] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.858788 22464 ts_tablet_manager.cc:1428] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 02:45:26.865072 22459 tablet_bootstrap.cc:492] T 62b1e9f4216845739fc757c975eb8c02 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:26.865259 22459 ts_tablet_manager.cc:1397] T 62b1e9f4216845739fc757c975eb8c02 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.042s user 0.003s sys 0.002s
I20260806 02:45:26.866356 22459 raft_consensus.cc:348] T 62b1e9f4216845739fc757c975eb8c02 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.866678 22459 raft_consensus.cc:374] T 62b1e9f4216845739fc757c975eb8c02 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.866860 22459 raft_consensus.cc:729] T 62b1e9f4216845739fc757c975eb8c02 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.867359 22459 consensus_queue.cc:260] T 62b1e9f4216845739fc757c975eb8c02 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.872618 21269 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Sending DeleteTablet for 3 replicas of tablet 3b3a4c79185241a7a8522135f0a513b3
I20260806 02:45:26.873656 21494 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b3a4c79185241a7a8522135f0a513b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 178955fff8a44eb7b629f09adff0d456) from {username='slave'} at 127.0.0.1:33118
I20260806 02:45:26.874223 21863 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b3a4c79185241a7a8522135f0a513b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 178955fff8a44eb7b629f09adff0d456) from {username='slave'} at 127.0.0.1:56374
I20260806 02:45:26.874449 22464 tablet_bootstrap.cc:492] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:26.874496 22448 tablet_replica.cc:330] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:26.874773 22448 raft_consensus.cc:2229] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:26.875022 22448 raft_consensus.cc:2258] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:26.875054 21681 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b3a4c79185241a7a8522135f0a513b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 178955fff8a44eb7b629f09adff0d456) from {username='slave'} at 127.0.0.1:38406
I20260806 02:45:26.875735 22448 ts_tablet_manager.cc:1905] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:26.876176 22459 ts_tablet_manager.cc:1428] T 62b1e9f4216845739fc757c975eb8c02 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260806 02:45:26.876596 22459 tablet_bootstrap.cc:492] T aac61d5d826042e4bcebdcfde8b71a1e P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:26.881871 22444 tablet_replica.cc:330] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:26.882229 22444 raft_consensus.cc:2229] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:45:26.882551 22459 tablet_bootstrap.cc:654] T aac61d5d826042e4bcebdcfde8b71a1e P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.882697 22444 raft_consensus.cc:2258] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:26.883536 22444 ts_tablet_manager.cc:1905] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:26.883618 21269 catalog_manager.cc:6565] ReplaceTablet: tablet 3b3a4c79185241a7a8522135f0a513b3 deleted and replaced by tablet 178955fff8a44eb7b629f09adff0d456
I20260806 02:45:26.890153 22445 ts_tablet_manager.cc:1918] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:26.890362 22445 log.cc:1196] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/3f0c5aff461940d999d0ed14292db320
I20260806 02:45:26.890858 22445 ts_tablet_manager.cc:1939] T 3f0c5aff461940d999d0ed14292db320 P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:26.892102 22464 tablet_bootstrap.cc:654] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.892580 21494 tablet_service.cc:1467] Processing CreateTablet for tablet 178955fff8a44eb7b629f09adff0d456 (DEFAULT_TABLE table=default.table_c2d5cb4cc8a44eaca15ba851ecc366c8 [id=ccf2db5226644b498abe88bba95d6fef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.892601 21681 tablet_service.cc:1467] Processing CreateTablet for tablet 178955fff8a44eb7b629f09adff0d456 (DEFAULT_TABLE table=default.table_c2d5cb4cc8a44eaca15ba851ecc366c8 [id=ccf2db5226644b498abe88bba95d6fef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.893059 21681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 178955fff8a44eb7b629f09adff0d456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.893127 21494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 178955fff8a44eb7b629f09adff0d456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.893616 21863 tablet_service.cc:1467] Processing CreateTablet for tablet 178955fff8a44eb7b629f09adff0d456 (DEFAULT_TABLE table=default.table_c2d5cb4cc8a44eaca15ba851ecc366c8 [id=ccf2db5226644b498abe88bba95d6fef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:26.893986 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 178955fff8a44eb7b629f09adff0d456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:26.894479 21251 catalog_manager.cc:4841] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): tablet 3f0c5aff461940d999d0ed14292db320 (table default.table_a7715d8e001f48b2899c3c2a56a18a06 [id=21d847eb419e4c0998a01f87c8cd92da]) successfully deleted
I20260806 02:45:26.894896 22445 tablet_replica.cc:330] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:26.895157 22445 raft_consensus.cc:2229] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:26.902271 22463 tablet_bootstrap.cc:492] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:26.902649 22463 ts_tablet_manager.cc:1397] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.066s user 0.005s sys 0.005s
I20260806 02:45:26.904464 22463 raft_consensus.cc:348] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.904919 22463 raft_consensus.cc:374] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.905095 22463 raft_consensus.cc:729] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.895401 22445 raft_consensus.cc:2258] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:26.905722 22463 consensus_queue.cc:260] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.906169 22445 ts_tablet_manager.cc:1905] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:26.907876 22463 ts_tablet_manager.cc:1428] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260806 02:45:26.910555 21269 catalog_manager.cc:5587] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593) does not have the latest schema for tablet 62b1e9f4216845739fc757c975eb8c02 (table default.table_8797f6f61c444bd0a4e4555f6ca2fe30 [id=370d5895c35f4c96878270e71cac2443]). Expected version 1 got 0
I20260806 02:45:26.911123 22407 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:26.911459 22407 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.912264 22407 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:26.912941 22463 tablet_bootstrap.cc:492] T aac61d5d826042e4bcebdcfde8b71a1e P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:26.913132 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba32fbee948d4adf96164d36673a1db6" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:26.913602 21888 raft_consensus.cc:2454] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:26.914127 21617 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:26.914402 22407 raft_consensus.cc:2781] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:26.914675 22407 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:26.914913 22407 raft_consensus.cc:3037] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:26.915023 22445 ts_tablet_manager.cc:1918] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:26.915184 22445 log.cc:1196] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/3b3a4c79185241a7a8522135f0a513b3
I20260806 02:45:26.915685 22445 ts_tablet_manager.cc:1939] T 3b3a4c79185241a7a8522135f0a513b3 P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:26.917004 21251 catalog_manager.cc:4841] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): tablet 3b3a4c79185241a7a8522135f0a513b3 (table default.table_c2d5cb4cc8a44eaca15ba851ecc366c8 [id=ccf2db5226644b498abe88bba95d6fef]) successfully deleted
W20260806 02:45:26.917200 21269 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 62b1e9f4216845739fc757c975eb8c02 (table default.table_8797f6f61c444bd0a4e4555f6ca2fe30 [id=370d5895c35f4c96878270e71cac2443], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 62b1e9f4216845739fc757c975eb8c02 consensus state has no leader
I20260806 02:45:26.917850 22459 tablet_bootstrap.cc:492] T aac61d5d826042e4bcebdcfde8b71a1e P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:26.918176 22459 ts_tablet_manager.cc:1397] T aac61d5d826042e4bcebdcfde8b71a1e P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.042s user 0.005s sys 0.001s
I20260806 02:45:26.918503 22444 ts_tablet_manager.cc:1918] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:26.921787 21520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba32fbee948d4adf96164d36673a1db6" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:26.922151 21520 raft_consensus.cc:2454] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:26.922691 22459 raft_consensus.cc:348] T aac61d5d826042e4bcebdcfde8b71a1e P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:26.923029 22459 raft_consensus.cc:374] T aac61d5d826042e4bcebdcfde8b71a1e P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.923270 22459 raft_consensus.cc:729] T aac61d5d826042e4bcebdcfde8b71a1e P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.923583 22459 consensus_queue.cc:260] T aac61d5d826042e4bcebdcfde8b71a1e P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:26.924589 22459 ts_tablet_manager.cc:1428] T aac61d5d826042e4bcebdcfde8b71a1e P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.006s user 0.001s sys 0.002s
I20260806 02:45:26.925076 21269 catalog_manager.cc:5587] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655) does not have the latest schema for tablet 62b1e9f4216845739fc757c975eb8c02 (table default.table_8797f6f61c444bd0a4e4555f6ca2fe30 [id=370d5895c35f4c96878270e71cac2443]). Expected version 1 got 0
I20260806 02:45:26.925823 22448 ts_tablet_manager.cc:1918] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:26.926056 22448 log.cc:1196] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/3b3a4c79185241a7a8522135f0a513b3
I20260806 02:45:26.926594 22448 ts_tablet_manager.cc:1939] T 3b3a4c79185241a7a8522135f0a513b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:26.928561 21269 master_service.cc:946] ReplaceTablet: received request to replace tablet b4e3f2c8eb2c4ea8a1b041be073bdd08 from {username='slave'} at 127.0.0.1:46462
I20260806 02:45:26.930136 22407 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.930938 22407 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:26.933974 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba32fbee948d4adf96164d36673a1db6" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:26.934368 21888 raft_consensus.cc:3037] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:26.935482 22463 tablet_bootstrap.cc:654] T aac61d5d826042e4bcebdcfde8b71a1e P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.936872 22444 log.cc:1196] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/3b3a4c79185241a7a8522135f0a513b3
I20260806 02:45:26.937543 22444 ts_tablet_manager.cc:1939] T 3b3a4c79185241a7a8522135f0a513b3 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:26.939008 21255 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet 3b3a4c79185241a7a8522135f0a513b3 (table default.table_c2d5cb4cc8a44eaca15ba851ecc366c8 [id=ccf2db5226644b498abe88bba95d6fef]) successfully deleted
I20260806 02:45:26.940008 21269 catalog_manager.cc:5587] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655) does not have the latest schema for tablet 62b1e9f4216845739fc757c975eb8c02 (table default.table_8797f6f61c444bd0a4e4555f6ca2fe30 [id=370d5895c35f4c96878270e71cac2443]). Expected version 1 got 0
I20260806 02:45:26.940687 21520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba32fbee948d4adf96164d36673a1db6" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:26.941093 21520 raft_consensus.cc:3037] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:26.941910 21269 catalog_manager.cc:5587] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593) does not have the latest schema for tablet 62b1e9f4216845739fc757c975eb8c02 (table default.table_8797f6f61c444bd0a4e4555f6ca2fe30 [id=370d5895c35f4c96878270e71cac2443]). Expected version 1 got 0
I20260806 02:45:26.942137 21888 raft_consensus.cc:2454] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:26.942615 21617 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:26.943305 22407 raft_consensus.cc:2781] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:26.943645 22407 raft_consensus.cc:686] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Becoming Leader. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Running, Role: LEADER
I20260806 02:45:26.944156 22407 consensus_queue.cc:237] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.944450 21520 raft_consensus.cc:2454] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:26.944607 22459 tablet_bootstrap.cc:492] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:26.949433 22466 raft_consensus.cc:482] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:26.949643 22466 raft_consensus.cc:504] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.950691 22466 leader_election.cc:288] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:26.954949 21705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81dd98ae43b7429992d48eeaa9f17a70" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" is_pre_election: true
I20260806 02:45:26.954995 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81dd98ae43b7429992d48eeaa9f17a70" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:26.955331 21888 raft_consensus.cc:2454] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 0.
I20260806 02:45:26.955380 21705 raft_consensus.cc:2454] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 0.
I20260806 02:45:26.955956 21434 leader_election.cc:302] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:26.956524 22466 raft_consensus.cc:2781] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:26.956753 22466 raft_consensus.cc:482] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:26.956874 22466 raft_consensus.cc:3037] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:26.960747 22466 raft_consensus.cc:504] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.962190 21705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81dd98ae43b7429992d48eeaa9f17a70" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
I20260806 02:45:26.962446 21705 raft_consensus.cc:3037] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:26.963825 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81dd98ae43b7429992d48eeaa9f17a70" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:26.964154 21888 raft_consensus.cc:3037] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:26.965073 22464 tablet_bootstrap.cc:492] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:26.965258 22464 ts_tablet_manager.cc:1397] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.105s user 0.012s sys 0.007s
I20260806 02:45:26.966320 21705 raft_consensus.cc:2454] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 1.
I20260806 02:45:26.966362 22464 raft_consensus.cc:348] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.966785 22464 raft_consensus.cc:374] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:26.966923 21436 leader_election.cc:302] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f; no voters:
I20260806 02:45:26.967056 22464 raft_consensus.cc:729] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:26.967082 22466 leader_election.cc:288] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 election: Requested vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:26.967731 22466 raft_consensus.cc:2781] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:26.967952 22466 raft_consensus.cc:686] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 LEADER]: Becoming Leader. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Running, Role: LEADER
I20260806 02:45:26.968351 22466 consensus_queue.cc:237] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.967437 22464 consensus_queue.cc:260] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:26.974658 22464 ts_tablet_manager.cc:1428] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.009s user 0.004s sys 0.000s
I20260806 02:45:26.974926 21269 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46462:
table { table_name: "default.Table_0e9bb7fa1ed44ca0af8546b2c255fa62" } modify_external_catalogs: true
W20260806 02:45:26.975692 21253 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 62b1e9f4216845739fc757c975eb8c02 (table default.table_8797f6f61c444bd0a4e4555f6ca2fe30 [id=370d5895c35f4c96878270e71cac2443], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 62b1e9f4216845739fc757c975eb8c02 consensus state has no leader
I20260806 02:45:26.977386 21269 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46462:
table { table_name: "default.Table_0e9bb7fa1ed44ca0af8546b2c255fa62" } modify_external_catalogs: true
I20260806 02:45:26.977967 22459 tablet_bootstrap.cc:654] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:26.978529 22464 tablet_bootstrap.cc:492] T 62b1e9f4216845739fc757c975eb8c02 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:26.981264 21888 raft_consensus.cc:2454] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 1.
I20260806 02:45:26.982213 22464 tablet_bootstrap.cc:654] T 62b1e9f4216845739fc757c975eb8c02 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:27.009608 22459 tablet_bootstrap.cc:492] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:27.010133 22459 ts_tablet_manager.cc:1397] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.065s user 0.002s sys 0.007s
I20260806 02:45:27.010486 22463 tablet_bootstrap.cc:492] T aac61d5d826042e4bcebdcfde8b71a1e P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:27.010738 22463 ts_tablet_manager.cc:1397] T aac61d5d826042e4bcebdcfde8b71a1e P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.102s user 0.004s sys 0.008s
I20260806 02:45:27.011585 22464 tablet_bootstrap.cc:492] T 62b1e9f4216845739fc757c975eb8c02 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:27.011664 22463 raft_consensus.cc:348] T aac61d5d826042e4bcebdcfde8b71a1e P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.012009 22463 raft_consensus.cc:374] T aac61d5d826042e4bcebdcfde8b71a1e P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.012159 22463 raft_consensus.cc:729] T aac61d5d826042e4bcebdcfde8b71a1e P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.012198 22464 ts_tablet_manager.cc:1397] T 62b1e9f4216845739fc757c975eb8c02 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.034s user 0.001s sys 0.004s
I20260806 02:45:27.012499 22463 consensus_queue.cc:260] T aac61d5d826042e4bcebdcfde8b71a1e P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.013283 22464 raft_consensus.cc:348] T 62b1e9f4216845739fc757c975eb8c02 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.013682 22464 raft_consensus.cc:374] T 62b1e9f4216845739fc757c975eb8c02 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.013805 22464 raft_consensus.cc:729] T 62b1e9f4216845739fc757c975eb8c02 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.015198 22463 ts_tablet_manager.cc:1428] T aac61d5d826042e4bcebdcfde8b71a1e P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260806 02:45:27.015599 22463 tablet_bootstrap.cc:492] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:27.017949 22464 consensus_queue.cc:260] T 62b1e9f4216845739fc757c975eb8c02 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.017990 22459 raft_consensus.cc:348] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.018528 22459 raft_consensus.cc:374] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.018764 22459 raft_consensus.cc:729] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.019168 22459 consensus_queue.cc:260] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.020406 22463 tablet_bootstrap.cc:654] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:27.025017 22464 ts_tablet_manager.cc:1428] T 62b1e9f4216845739fc757c975eb8c02 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.013s user 0.003s sys 0.000s
I20260806 02:45:27.026043 22459 ts_tablet_manager.cc:1428] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.014s user 0.000s sys 0.003s
I20260806 02:45:27.029424 22464 tablet_bootstrap.cc:492] T aac61d5d826042e4bcebdcfde8b71a1e P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:27.034031 22464 tablet_bootstrap.cc:654] T aac61d5d826042e4bcebdcfde8b71a1e P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:27.035734 21251 catalog_manager.cc:4841] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): tablet 3b3a4c79185241a7a8522135f0a513b3 (table default.table_c2d5cb4cc8a44eaca15ba851ecc366c8 [id=ccf2db5226644b498abe88bba95d6fef]) successfully deleted
I20260806 02:45:27.042862 22463 tablet_bootstrap.cc:492] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:27.043175 22463 ts_tablet_manager.cc:1397] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.028s user 0.005s sys 0.002s
I20260806 02:45:27.044169 22463 raft_consensus.cc:348] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.044528 22463 raft_consensus.cc:374] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.044636 22463 raft_consensus.cc:729] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.044977 22463 consensus_queue.cc:260] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.046525 22464 tablet_bootstrap.cc:492] T aac61d5d826042e4bcebdcfde8b71a1e P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:27.046753 22464 ts_tablet_manager.cc:1397] T aac61d5d826042e4bcebdcfde8b71a1e P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.007s
I20260806 02:45:27.048002 22464 raft_consensus.cc:348] T aac61d5d826042e4bcebdcfde8b71a1e P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.048333 22464 raft_consensus.cc:374] T aac61d5d826042e4bcebdcfde8b71a1e P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.048470 22464 raft_consensus.cc:729] T aac61d5d826042e4bcebdcfde8b71a1e P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.048780 22464 consensus_queue.cc:260] T aac61d5d826042e4bcebdcfde8b71a1e P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.049861 22464 ts_tablet_manager.cc:1428] T aac61d5d826042e4bcebdcfde8b71a1e P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 02:45:27.050801 22464 tablet_bootstrap.cc:492] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:27.055768 22463 ts_tablet_manager.cc:1428] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.012s user 0.002s sys 0.002s
I20260806 02:45:27.058178 22464 tablet_bootstrap.cc:654] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:27.069412 22464 tablet_bootstrap.cc:492] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:27.069826 22464 ts_tablet_manager.cc:1397] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.003s
I20260806 02:45:27.070930 22464 raft_consensus.cc:348] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.071349 22464 raft_consensus.cc:374] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.071527 22464 raft_consensus.cc:729] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.071875 22464 consensus_queue.cc:260] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.074167 21268 catalog_manager.cc:3411] renamed table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022] in HMS: new name default.table_8e31b0866b344f85b9631a2e31b68af1
I20260806 02:45:27.077327 22464 ts_tablet_manager.cc:1428] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260806 02:45:27.115406 22407 raft_consensus.cc:482] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:27.115732 22407 raft_consensus.cc:504] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.116165 21264 catalog_manager.cc:2148] created HMS catalog entry for table [id=f6939f0d45ae4f358d206a8cf2fe5db0]
I20260806 02:45:27.116536 22407 leader_election.cc:288] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:27.117585 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "178955fff8a44eb7b629f09adff0d456" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:27.117607 21520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "178955fff8a44eb7b629f09adff0d456" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:27.117868 21888 raft_consensus.cc:2454] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:27.117908 21520 raft_consensus.cc:2454] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:27.118513 21617 leader_election.cc:302] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f; no voters:
I20260806 02:45:27.119279 22407 raft_consensus.cc:2781] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:27.119395 22407 raft_consensus.cc:482] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:27.119619 22407 raft_consensus.cc:3037] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.122673 22407 raft_consensus.cc:504] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.124428 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "178955fff8a44eb7b629f09adff0d456" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:27.123574 22407 leader_election.cc:288] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Requested vote from peers 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:27.124750 21888 raft_consensus.cc:3037] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.125007 21520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "178955fff8a44eb7b629f09adff0d456" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:27.125398 21520 raft_consensus.cc:3037] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.128437 21888 raft_consensus.cc:2454] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:27.128708 21264 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46462 (request call id 14) took 1140 ms. Trace:
I20260806 02:45:27.129027 21617 leader_election.cc:302] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:27.129594 22407 raft_consensus.cc:2781] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:27.130317 22407 raft_consensus.cc:686] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Becoming Leader. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Running, Role: LEADER
I20260806 02:45:27.128913 21264 rpcz_store.cc:276] 0806 02:45:25.987782 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.987845 (+ 63us) service_pool.cc:227] Handling call
0806 02:45:26.288732 (+300887us) catalog_manager.cc:2075] Acquired catalog manager lock
0806 02:45:26.289327 (+ 595us) catalog_manager.cc:2121] Created new table and tablet info
0806 02:45:27.116096 (+826769us) catalog_manager.cc:2147] Created new table in HMS catalog
0806 02:45:27.128159 (+ 12063us) catalog_manager.cc:2181] Wrote table and tablets to system table
0806 02:45:27.128375 (+ 216us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0806 02:45:27.128529 (+ 154us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0806 02:45:27.117234 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0806 02:45:27.117438 (+ 204us) write_op.cc:432] Acquiring schema lock in shared mode
0806 02:45:27.117445 (+ 7us) write_op.cc:435] Acquired schema lock
0806 02:45:27.117447 (+ 2us) tablet.cc:660] Decoding operations
0806 02:45:27.117608 (+ 161us) write_op.cc:620] Acquiring the partition lock for write op
0806 02:45:27.117627 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0806 02:45:27.117635 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0806 02:45:27.117735 (+ 100us) tablet.cc:699] Row locks acquired
0806 02:45:27.117737 (+ 2us) write_op.cc:260] PREPARE: finished
0806 02:45:27.117784 (+ 47us) write_op.cc:270] Start()
0806 02:45:27.117828 (+ 44us) write_op.cc:276] Timestamp: P: 1785984327117748 usec, L: 0
0806 02:45:27.117831 (+ 3us) op_driver.cc:331] REPLICATION: starting
0806 02:45:27.120030 (+ 2199us) log.cc:842] Serialized 432 byte log entry
0806 02:45:27.127383 (+ 7353us) op_driver.cc:444] REPLICATION: finished
0806 02:45:27.127547 (+ 164us) write_op.cc:301] APPLY: starting
0806 02:45:27.127566 (+ 19us) tablet.cc:1365] starting BulkCheckPresence
0806 02:45:27.127615 (+ 49us) tablet.cc:1368] finished BulkCheckPresence
0806 02:45:27.127653 (+ 38us) tablet.cc:1370] starting ApplyRowOperation cycle
0806 02:45:27.127777 (+ 124us) tablet.cc:1381] finished ApplyRowOperation cycle
0806 02:45:27.127827 (+ 50us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0806 02:45:27.127834 (+ 7us) write_op.cc:312] APPLY: finished
0806 02:45:27.127920 (+ 86us) log.cc:842] Serialized 28 byte log entry
0806 02:45:27.127994 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:27.128016 (+ 22us) write_op.cc:454] Released schema lock
0806 02:45:27.128025 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0806 02:45:27.128088 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:27.128091 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":763302,"Hive Metastore.run_cpu_time_us":302,"Hive Metastore.run_wall_time_us":42110,"child_traces":[["op",{"apply.queue_time_us":86,"apply.run_cpu_time_us":622,"apply.run_wall_time_us":621,"mutex_wait_us":4283,"num_ops":2,"prepare.queue_time_us":67,"prepare.run_cpu_time_us":1166,"prepare.run_wall_time_us":3075,"raft.queue_time_us":219,"raft.run_cpu_time_us":1040,"raft.run_wall_time_us":3202,"replication_time_us":9524,"spinlock_wait_cycles":3654400}]]}
I20260806 02:45:27.130611 22407 consensus_queue.cc:237] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.134230 21520 raft_consensus.cc:2454] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:27.146981 21494 tablet_service.cc:1467] Processing CreateTablet for tablet fdc39b7dea2c43cebc730cffacd38d10 (DEFAULT_TABLE table=default.table_c5be13d5ad2c4a479d2cc9074d14038f [id=f6939f0d45ae4f358d206a8cf2fe5db0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:27.147538 21494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdc39b7dea2c43cebc730cffacd38d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:27.147529 21863 tablet_service.cc:1467] Processing CreateTablet for tablet fdc39b7dea2c43cebc730cffacd38d10 (DEFAULT_TABLE table=default.table_c5be13d5ad2c4a479d2cc9074d14038f [id=f6939f0d45ae4f358d206a8cf2fe5db0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:27.148065 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdc39b7dea2c43cebc730cffacd38d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:27.148250 21264 catalog_manager.cc:5495] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f reported cstate change: term changed from 0 to 1, leader changed from <none> to 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2). New cstate: current_term: 1 leader_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260806 02:45:27.150030 21681 tablet_service.cc:1467] Processing CreateTablet for tablet fdc39b7dea2c43cebc730cffacd38d10 (DEFAULT_TABLE table=default.table_c5be13d5ad2c4a479d2cc9074d14038f [id=f6939f0d45ae4f358d206a8cf2fe5db0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:27.150622 21681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdc39b7dea2c43cebc730cffacd38d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:27.155126 22459 tablet_bootstrap.cc:492] T fdc39b7dea2c43cebc730cffacd38d10 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:27.155660 22464 tablet_bootstrap.cc:492] T fdc39b7dea2c43cebc730cffacd38d10 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:27.157584 22459 tablet_bootstrap.cc:654] T fdc39b7dea2c43cebc730cffacd38d10 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:27.157819 22464 tablet_bootstrap.cc:654] T fdc39b7dea2c43cebc730cffacd38d10 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:27.163574 22464 tablet_bootstrap.cc:492] T fdc39b7dea2c43cebc730cffacd38d10 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:27.163771 22464 ts_tablet_manager.cc:1397] T fdc39b7dea2c43cebc730cffacd38d10 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260806 02:45:27.164637 22464 raft_consensus.cc:348] T fdc39b7dea2c43cebc730cffacd38d10 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.164999 22464 raft_consensus.cc:374] T fdc39b7dea2c43cebc730cffacd38d10 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.165278 22464 raft_consensus.cc:729] T fdc39b7dea2c43cebc730cffacd38d10 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.165732 22464 consensus_queue.cc:260] T fdc39b7dea2c43cebc730cffacd38d10 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.168316 21264 catalog_manager.cc:5495] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1). New cstate: current_term: 1 leader_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260806 02:45:27.171710 22464 ts_tablet_manager.cc:1428] T fdc39b7dea2c43cebc730cffacd38d10 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260806 02:45:27.180341 21263 catalog_manager.cc:2148] created HMS catalog entry for table [id=ba73c7eac7c245e598f7595efbc1e42c]
I20260806 02:45:27.186851 21264 catalog_manager.cc:5587] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883) does not have the latest schema for tablet 62b1e9f4216845739fc757c975eb8c02 (table default.table_8797f6f61c444bd0a4e4555f6ca2fe30 [id=370d5895c35f4c96878270e71cac2443]). Expected version 1 got 0
I20260806 02:45:27.191915 21263 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46462 (request call id 15) took 1202 ms. Trace:
I20260806 02:45:27.191653 21264 catalog_manager.cc:5495] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f reported cstate change: term changed from 0 to 1, leader changed from <none> to 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2). New cstate: current_term: 1 leader_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: HEALTHY } } }
I20260806 02:45:27.192199 21263 rpcz_store.cc:276] 0806 02:45:25.989689 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:25.989754 (+ 65us) service_pool.cc:227] Handling call
0806 02:45:26.289902 (+300148us) catalog_manager.cc:2075] Acquired catalog manager lock
0806 02:45:26.290438 (+ 536us) catalog_manager.cc:2121] Created new table and tablet info
0806 02:45:26.310843 (+ 20405us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x6160000161c0. stack: 00007fe39a4e766e 00007fe39a4e7430 00007fe39956e5ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007fe3ad3172ec 00007fe3ad31cc67 00007fe3ad51d46a 00007fe3a480f466 00007fe3a480f195 00007fe3a10239be 00007fe3a102279e 00007fe3a1028b25 00007fe3a102954f 00007fe3a1029378
0806 02:45:27.180279 (+869436us) catalog_manager.cc:2147] Created new table in HMS catalog
0806 02:45:27.191510 (+ 11231us) catalog_manager.cc:2181] Wrote table and tablets to system table
0806 02:45:27.191641 (+ 131us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0806 02:45:27.191756 (+ 115us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0806 02:45:27.181727 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0806 02:45:27.181841 (+ 114us) write_op.cc:432] Acquiring schema lock in shared mode
0806 02:45:27.181847 (+ 6us) write_op.cc:435] Acquired schema lock
0806 02:45:27.181850 (+ 3us) tablet.cc:660] Decoding operations
0806 02:45:27.184041 (+ 2191us) write_op.cc:620] Acquiring the partition lock for write op
0806 02:45:27.184055 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0806 02:45:27.184061 (+ 6us) tablet.cc:683] Acquiring locks for 2 operations
0806 02:45:27.184178 (+ 117us) tablet.cc:699] Row locks acquired
0806 02:45:27.184181 (+ 3us) write_op.cc:260] PREPARE: finished
0806 02:45:27.184221 (+ 40us) write_op.cc:270] Start()
0806 02:45:27.184258 (+ 37us) write_op.cc:276] Timestamp: P: 1785984327184182 usec, L: 0
0806 02:45:27.184263 (+ 5us) op_driver.cc:331] REPLICATION: starting
0806 02:45:27.184528 (+ 265us) log.cc:842] Serialized 432 byte log entry
0806 02:45:27.188696 (+ 4168us) op_driver.cc:444] REPLICATION: finished
0806 02:45:27.190787 (+ 2091us) write_op.cc:301] APPLY: starting
0806 02:45:27.190804 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
0806 02:45:27.190852 (+ 48us) tablet.cc:1368] finished BulkCheckPresence
0806 02:45:27.190879 (+ 27us) tablet.cc:1370] starting ApplyRowOperation cycle
0806 02:45:27.191009 (+ 130us) tablet.cc:1381] finished ApplyRowOperation cycle
0806 02:45:27.191046 (+ 37us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0806 02:45:27.191053 (+ 7us) write_op.cc:312] APPLY: finished
0806 02:45:27.191208 (+ 155us) log.cc:842] Serialized 28 byte log entry
0806 02:45:27.191284 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:27.191306 (+ 22us) write_op.cc:454] Released schema lock
0806 02:45:27.191313 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0806 02:45:27.191381 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:27.191385 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":805086,"Hive Metastore.run_cpu_time_us":343,"Hive Metastore.run_wall_time_us":63931,"spinlock_wait_cycles":46373248,"child_traces":[["op",{"apply.queue_time_us":2027,"apply.run_cpu_time_us":661,"apply.run_wall_time_us":660,"num_ops":2,"prepare.queue_time_us":645,"prepare.run_cpu_time_us":814,"prepare.run_wall_time_us":2893,"replication_time_us":4408}]]}
I20260806 02:45:27.205626 22463 tablet_bootstrap.cc:492] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:27.208395 22463 tablet_bootstrap.cc:654] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:27.208446 21263 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46462:
name: "default.Table_2b3ca9846e244fc593dc07fd72a58f3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:27.210554 21264 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46462:
table { table_name: "default.Table_e8df6dde49954df7b57567fc0cd12872" } modify_external_catalogs: true
I20260806 02:45:27.211015 21264 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46462:
table { table_name: "default.Table_e8df6dde49954df7b57567fc0cd12872" } modify_external_catalogs: true
I20260806 02:45:27.212672 22459 tablet_bootstrap.cc:492] T fdc39b7dea2c43cebc730cffacd38d10 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:27.212949 22459 ts_tablet_manager.cc:1397] T fdc39b7dea2c43cebc730cffacd38d10 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.058s user 0.006s sys 0.010s
I20260806 02:45:27.214143 22459 raft_consensus.cc:348] T fdc39b7dea2c43cebc730cffacd38d10 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.214568 22459 raft_consensus.cc:374] T fdc39b7dea2c43cebc730cffacd38d10 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.214735 22459 raft_consensus.cc:729] T fdc39b7dea2c43cebc730cffacd38d10 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.215116 22459 consensus_queue.cc:260] T fdc39b7dea2c43cebc730cffacd38d10 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.220400 21681 tablet_service.cc:1467] Processing CreateTablet for tablet ca768d858ace4b2b900648ac35f91ecd (DEFAULT_TABLE table=default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:27.220468 21494 tablet_service.cc:1467] Processing CreateTablet for tablet ca768d858ace4b2b900648ac35f91ecd (DEFAULT_TABLE table=default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:27.221002 21494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca768d858ace4b2b900648ac35f91ecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:27.221024 21681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca768d858ace4b2b900648ac35f91ecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:27.223309 21863 tablet_service.cc:1467] Processing CreateTablet for tablet ca768d858ace4b2b900648ac35f91ecd (DEFAULT_TABLE table=default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:27.223817 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca768d858ace4b2b900648ac35f91ecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:27.226704 22463 tablet_bootstrap.cc:492] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:27.226883 22463 ts_tablet_manager.cc:1397] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.010s
I20260806 02:45:27.228016 22463 raft_consensus.cc:348] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.228364 22463 raft_consensus.cc:374] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.228639 22463 raft_consensus.cc:729] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.229121 22463 consensus_queue.cc:260] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.230203 22463 ts_tablet_manager.cc:1428] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260806 02:45:27.230516 22463 tablet_bootstrap.cc:492] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:27.235082 22463 tablet_bootstrap.cc:654] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:27.236629 22459 ts_tablet_manager.cc:1428] T fdc39b7dea2c43cebc730cffacd38d10 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.009s user 0.004s sys 0.000s
I20260806 02:45:27.237053 22459 tablet_bootstrap.cc:492] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:27.239691 22459 tablet_bootstrap.cc:654] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:27.243016 22407 raft_consensus.cc:482] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:27.243188 22407 raft_consensus.cc:504] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.243883 22407 leader_election.cc:288] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:27.244401 22464 tablet_bootstrap.cc:492] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:27.245213 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62b1e9f4216845739fc757c975eb8c02" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:27.245592 21888 raft_consensus.cc:2454] T 62b1e9f4216845739fc757c975eb8c02 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:27.246050 22463 tablet_bootstrap.cc:492] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:27.246169 21617 leader_election.cc:302] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:27.246309 22463 ts_tablet_manager.cc:1397] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.007s
I20260806 02:45:27.246515 22464 tablet_bootstrap.cc:654] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:27.247462 22463 raft_consensus.cc:348] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.247877 22463 raft_consensus.cc:374] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.247967 22463 raft_consensus.cc:729] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.248394 22463 consensus_queue.cc:260] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.248773 22497 raft_consensus.cc:2781] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:27.248872 22497 raft_consensus.cc:482] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:27.248991 22497 raft_consensus.cc:3037] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.248914 21520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62b1e9f4216845739fc757c975eb8c02" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:27.249284 21520 raft_consensus.cc:2454] T 62b1e9f4216845739fc757c975eb8c02 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:27.253248 22497 raft_consensus.cc:504] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.257745 22497 leader_election.cc:288] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:27.258411 22463 ts_tablet_manager.cc:1428] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.012s user 0.002s sys 0.001s
I20260806 02:45:27.260459 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62b1e9f4216845739fc757c975eb8c02" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:27.260785 21888 raft_consensus.cc:3037] T 62b1e9f4216845739fc757c975eb8c02 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.263751 21520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62b1e9f4216845739fc757c975eb8c02" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:27.264149 21520 raft_consensus.cc:3037] T 62b1e9f4216845739fc757c975eb8c02 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.264276 21888 raft_consensus.cc:2454] T 62b1e9f4216845739fc757c975eb8c02 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:27.264878 21617 leader_election.cc:302] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:27.265292 22497 raft_consensus.cc:2781] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:27.265553 22497 raft_consensus.cc:686] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Becoming Leader. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Running, Role: LEADER
I20260806 02:45:27.265908 22464 tablet_bootstrap.cc:492] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:27.266088 22464 ts_tablet_manager.cc:1397] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.022s user 0.011s sys 0.001s
I20260806 02:45:27.265938 22497 consensus_queue.cc:237] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.267557 22464 raft_consensus.cc:348] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.267875 22464 raft_consensus.cc:374] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.267984 22464 raft_consensus.cc:729] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.268271 22464 consensus_queue.cc:260] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.268847 22464 ts_tablet_manager.cc:1428] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 02:45:27.270370 21520 raft_consensus.cc:2454] T 62b1e9f4216845739fc757c975eb8c02 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:27.271523 22459 tablet_bootstrap.cc:492] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:27.271823 22459 ts_tablet_manager.cc:1397] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.016s
I20260806 02:45:27.274549 22459 raft_consensus.cc:348] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.276084 22459 raft_consensus.cc:374] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.276171 22459 raft_consensus.cc:729] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.276516 22459 consensus_queue.cc:260] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.280891 22459 ts_tablet_manager.cc:1428] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.009s user 0.000s sys 0.003s
I20260806 02:45:27.284410 22497 raft_consensus.cc:482] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:27.284698 22497 raft_consensus.cc:504] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.285501 22489 raft_consensus.cc:482] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:27.285931 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdc39b7dea2c43cebc730cffacd38d10" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:27.285837 22489 raft_consensus.cc:504] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.286216 21888 raft_consensus.cc:2454] T fdc39b7dea2c43cebc730cffacd38d10 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:27.286724 21617 leader_election.cc:302] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:27.286866 22489 leader_election.cc:288] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:27.287253 22497 leader_election.cc:288] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:27.287421 22497 raft_consensus.cc:2781] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:27.287525 22497 raft_consensus.cc:482] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:27.287619 22497 raft_consensus.cc:3037] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.287745 21705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9fd42a920b24117a4d687fec13ad17b" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" is_pre_election: true
I20260806 02:45:27.288070 21705 raft_consensus.cc:2454] T f9fd42a920b24117a4d687fec13ad17b P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:27.288405 21520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdc39b7dea2c43cebc730cffacd38d10" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:27.288654 21520 raft_consensus.cc:2454] T fdc39b7dea2c43cebc730cffacd38d10 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:27.288656 21806 leader_election.cc:302] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:27.289601 21520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9fd42a920b24117a4d687fec13ad17b" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:27.289875 21520 raft_consensus.cc:2454] T f9fd42a920b24117a4d687fec13ad17b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:27.290256 22489 raft_consensus.cc:2781] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:27.290526 22489 raft_consensus.cc:482] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:27.290681 22489 raft_consensus.cc:3037] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.293757 21266 catalog_manager.cc:2148] created HMS catalog entry for table [id=ee80e9ed535b43aaa5bd1dd782332704]
I20260806 02:45:27.294413 22489 raft_consensus.cc:504] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.295094 22489 leader_election.cc:288] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:27.295853 21520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9fd42a920b24117a4d687fec13ad17b" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:27.296072 21520 raft_consensus.cc:3037] T f9fd42a920b24117a4d687fec13ad17b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.299697 21520 raft_consensus.cc:2454] T f9fd42a920b24117a4d687fec13ad17b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:27.301368 21803 leader_election.cc:302] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:27.302359 22489 raft_consensus.cc:2781] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:27.302498 22489 raft_consensus.cc:686] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [term 1 LEADER]: Becoming Leader. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Running, Role: LEADER
I20260806 02:45:27.302698 21345 catalog_manager.cc:2709] Deleting table default.table_2b3ca9846e244fc593dc07fd72a58f3f [id=b4cb7469fede4d7cb3ae9bc653f5eec9] in response to Hive Metastore notification log event 22
I20260806 02:45:27.302767 22489 consensus_queue.cc:237] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
W20260806 02:45:27.303125 21345 hms_notification_log_listener.cc:294] Failed to handle Hive Metastore notification: 22 DROP_TABLE default.table_2b3ca9846e244fc593dc07fd72a58f3f: Not found: the table does not exist: table_id: "b4cb7469fede4d7cb3ae9bc653f5eec9" table_name: "default.table_2b3ca9846e244fc593dc07fd72a58f3f"
I20260806 02:45:27.304584 21345 catalog_manager.cc:2709] Deleting table default.table_cac77bd182314ee2a2068374ddeca8d4 [id=fe86c442832f49bc8efdb723bab0fc10] in response to Hive Metastore notification log event 23
I20260806 02:45:27.305040 21705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9fd42a920b24117a4d687fec13ad17b" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
W20260806 02:45:27.305186 21345 hms_notification_log_listener.cc:294] Failed to handle Hive Metastore notification: 23 DROP_TABLE default.table_cac77bd182314ee2a2068374ddeca8d4: Not found: the table does not exist: table_id: "fe86c442832f49bc8efdb723bab0fc10" table_name: "default.table_cac77bd182314ee2a2068374ddeca8d4"
I20260806 02:45:27.305481 21705 raft_consensus.cc:3037] T f9fd42a920b24117a4d687fec13ad17b P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.312824 22497 raft_consensus.cc:504] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.315091 21705 raft_consensus.cc:2454] T f9fd42a920b24117a4d687fec13ad17b P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:27.316526 21266 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46462 (request call id 19) took 1324 ms. Trace:
I20260806 02:45:27.316818 21266 rpcz_store.cc:276] 0806 02:45:25.992445 (+ 0us) service_pool.cc:170] Inserting onto call queue
0806 02:45:26.004493 (+ 12048us) service_pool.cc:227] Handling call
0806 02:45:26.291026 (+286533us) catalog_manager.cc:2075] Acquired catalog manager lock
0806 02:45:26.291523 (+ 497us) catalog_manager.cc:2121] Created new table and tablet info
0806 02:45:26.314230 (+ 22707us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x6160000161c0. stack: 00007fe39a4e766e 00007fe39a4e7430 00007fe39956e5ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007fe3ad3172ec 00007fe3ad31cc67 00007fe3ad51d46a 00007fe3a480f466 00007fe3a480f195 00007fe3a10239be 00007fe3a102279e 00007fe3a1028b25 00007fe3a102954f 00007fe3a1029378
0806 02:45:27.293633 (+979403us) catalog_manager.cc:2147] Created new table in HMS catalog
0806 02:45:27.316149 (+ 22516us) catalog_manager.cc:2181] Wrote table and tablets to system table
0806 02:45:27.316291 (+ 142us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0806 02:45:27.316416 (+ 125us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0806 02:45:27.294726 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0806 02:45:27.294891 (+ 165us) write_op.cc:432] Acquiring schema lock in shared mode
0806 02:45:27.294898 (+ 7us) write_op.cc:435] Acquired schema lock
0806 02:45:27.294900 (+ 2us) tablet.cc:660] Decoding operations
0806 02:45:27.295011 (+ 111us) write_op.cc:620] Acquiring the partition lock for write op
0806 02:45:27.295028 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0806 02:45:27.295033 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0806 02:45:27.295144 (+ 111us) tablet.cc:699] Row locks acquired
0806 02:45:27.295146 (+ 2us) write_op.cc:260] PREPARE: finished
0806 02:45:27.295195 (+ 49us) write_op.cc:270] Start()
0806 02:45:27.295239 (+ 44us) write_op.cc:276] Timestamp: P: 1785984327295141 usec, L: 0
0806 02:45:27.295246 (+ 7us) op_driver.cc:331] REPLICATION: starting
0806 02:45:27.295466 (+ 220us) log.cc:842] Serialized 432 byte log entry
0806 02:45:27.314610 (+ 19144us) op_driver.cc:444] REPLICATION: finished
0806 02:45:27.314738 (+ 128us) write_op.cc:301] APPLY: starting
0806 02:45:27.314756 (+ 18us) tablet.cc:1365] starting BulkCheckPresence
0806 02:45:27.314832 (+ 76us) tablet.cc:1368] finished BulkCheckPresence
0806 02:45:27.314862 (+ 30us) tablet.cc:1370] starting ApplyRowOperation cycle
0806 02:45:27.315010 (+ 148us) tablet.cc:1381] finished ApplyRowOperation cycle
0806 02:45:27.315054 (+ 44us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0806 02:45:27.315060 (+ 6us) write_op.cc:312] APPLY: finished
0806 02:45:27.315151 (+ 91us) log.cc:842] Serialized 28 byte log entry
0806 02:45:27.315231 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:27.315255 (+ 24us) write_op.cc:454] Released schema lock
0806 02:45:27.315262 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0806 02:45:27.315326 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0806 02:45:27.315330 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":865667,"Hive Metastore.run_cpu_time_us":307,"Hive Metastore.run_wall_time_us":113487,"spinlock_wait_cycles":51852032,"child_traces":[["op",{"apply.queue_time_us":60,"apply.run_cpu_time_us":647,"apply.run_wall_time_us":647,"num_ops":2,"prepare.queue_time_us":158,"prepare.run_cpu_time_us":887,"prepare.run_wall_time_us":888,"raft.queue_time_us":9554,"raft.run_cpu_time_us":574,"raft.run_wall_time_us":571,"replication_time_us":19338}]]}
I20260806 02:45:27.319001 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdc39b7dea2c43cebc730cffacd38d10" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:27.319355 21888 raft_consensus.cc:3037] T fdc39b7dea2c43cebc730cffacd38d10 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.325758 21888 raft_consensus.cc:2454] T fdc39b7dea2c43cebc730cffacd38d10 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:27.326828 21520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdc39b7dea2c43cebc730cffacd38d10" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:27.328869 21520 raft_consensus.cc:3037] T fdc39b7dea2c43cebc730cffacd38d10 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.330044 21617 leader_election.cc:302] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:27.330334 22497 leader_election.cc:288] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Requested vote from peers 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:27.330421 22407 raft_consensus.cc:2781] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:27.330753 22407 raft_consensus.cc:686] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Becoming Leader. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Running, Role: LEADER
I20260806 02:45:27.331037 22407 consensus_queue.cc:237] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.335747 21681 tablet_service.cc:1467] Processing CreateTablet for tablet bea6ade491c6486393aca875767dc1b3 (DEFAULT_TABLE table=default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:27.336227 21681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bea6ade491c6486393aca875767dc1b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:27.336409 21520 raft_consensus.cc:2454] T fdc39b7dea2c43cebc730cffacd38d10 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:27.339548 21266 master_service.cc:946] ReplaceTablet: received request to replace tablet d25a9c5a0123446dac00ad895381f96a from {username='slave'} at 127.0.0.1:46462
I20260806 02:45:27.343778 22463 tablet_bootstrap.cc:492] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:27.343761 21494 tablet_service.cc:1467] Processing CreateTablet for tablet bea6ade491c6486393aca875767dc1b3 (DEFAULT_TABLE table=default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:27.344007 21863 tablet_service.cc:1467] Processing CreateTablet for tablet bea6ade491c6486393aca875767dc1b3 (DEFAULT_TABLE table=default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:27.344328 21494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bea6ade491c6486393aca875767dc1b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:27.344534 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bea6ade491c6486393aca875767dc1b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:27.346035 22463 tablet_bootstrap.cc:654] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
W20260806 02:45:27.348600 21345 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 62b1e9f4216845739fc757c975eb8c02 consensus state has no leader
I20260806 02:45:27.352943 21345 catalog_manager.cc:2709] Deleting table default.table_a7715d8e001f48b2899c3c2a56a18a06 [id=21d847eb419e4c0998a01f87c8cd92da] in response to Hive Metastore notification log event 27
I20260806 02:45:27.363445 21266 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Sending DeleteTablet for 3 replicas of tablet d25a9c5a0123446dac00ad895381f96a
I20260806 02:45:27.364624 22459 tablet_bootstrap.cc:492] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:27.365052 22464 tablet_bootstrap.cc:492] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:27.365268 21266 catalog_manager.cc:6565] ReplaceTablet: tablet d25a9c5a0123446dac00ad895381f96a deleted and replaced by tablet 10189388393d47ce900b44cbad3ff425
I20260806 02:45:27.365203 21681 tablet_service.cc:1514] Processing DeleteTablet for tablet d25a9c5a0123446dac00ad895381f96a with delete_type TABLET_DATA_DELETED (replaced by tablet 10189388393d47ce900b44cbad3ff425) from {username='slave'} at 127.0.0.1:38406
I20260806 02:45:27.365653 22444 tablet_replica.cc:330] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:27.365950 22444 raft_consensus.cc:2229] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:45:27.366322 22444 raft_consensus.cc:2258] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:27.366820 22459 tablet_bootstrap.cc:654] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:27.366945 22444 ts_tablet_manager.cc:1905] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:27.367180 22464 tablet_bootstrap.cc:654] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:27.368176 21266 catalog_manager.cc:5495] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3). New cstate: current_term: 1 leader_uuid: "8ebd96b903b944f3b06851324b737bd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: HEALTHY } } }
I20260806 02:45:27.369052 21495 tablet_service.cc:1514] Processing DeleteTablet for tablet d25a9c5a0123446dac00ad895381f96a with delete_type TABLET_DATA_DELETED (replaced by tablet 10189388393d47ce900b44cbad3ff425) from {username='slave'} at 127.0.0.1:33118
I20260806 02:45:27.369438 22448 tablet_replica.cc:330] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:27.369692 22448 raft_consensus.cc:2229] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:27.369786 21865 tablet_service.cc:1514] Processing DeleteTablet for tablet d25a9c5a0123446dac00ad895381f96a with delete_type TABLET_DATA_DELETED (replaced by tablet 10189388393d47ce900b44cbad3ff425) from {username='slave'} at 127.0.0.1:56374
I20260806 02:45:27.369922 22448 raft_consensus.cc:2258] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:27.370124 22445 tablet_replica.cc:330] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:27.370352 22445 raft_consensus.cc:2229] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:27.370554 22448 ts_tablet_manager.cc:1905] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:27.370565 22445 raft_consensus.cc:2258] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:27.371083 22445 ts_tablet_manager.cc:1905] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:27.375483 22444 ts_tablet_manager.cc:1918] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:27.375761 22444 log.cc:1196] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/d25a9c5a0123446dac00ad895381f96a
I20260806 02:45:27.376506 22444 ts_tablet_manager.cc:1939] T d25a9c5a0123446dac00ad895381f96a P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:27.377038 22448 ts_tablet_manager.cc:1918] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:27.377193 22448 log.cc:1196] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/d25a9c5a0123446dac00ad895381f96a
I20260806 02:45:27.377939 21255 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet d25a9c5a0123446dac00ad895381f96a (table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e]) successfully deleted
I20260806 02:45:27.378453 22448 ts_tablet_manager.cc:1939] T d25a9c5a0123446dac00ad895381f96a P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:27.379505 21251 catalog_manager.cc:4841] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): tablet d25a9c5a0123446dac00ad895381f96a (table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e]) successfully deleted
I20260806 02:45:27.384059 22445 ts_tablet_manager.cc:1918] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:27.384212 22445 log.cc:1196] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/d25a9c5a0123446dac00ad895381f96a
I20260806 02:45:27.384366 22459 tablet_bootstrap.cc:492] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:27.384567 22459 ts_tablet_manager.cc:1397] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.000s
I20260806 02:45:27.384738 22445 ts_tablet_manager.cc:1939] T d25a9c5a0123446dac00ad895381f96a P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:27.385722 22459 raft_consensus.cc:348] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.386047 22459 raft_consensus.cc:374] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.386152 22459 raft_consensus.cc:729] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.386425 22459 consensus_queue.cc:260] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.388803 22459 ts_tablet_manager.cc:1428] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 02:45:27.393111 22464 tablet_bootstrap.cc:492] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:27.393289 22464 ts_tablet_manager.cc:1397] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.028s user 0.005s sys 0.004s
I20260806 02:45:27.394609 22464 raft_consensus.cc:348] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.394860 22464 raft_consensus.cc:374] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.395021 22464 raft_consensus.cc:729] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.395002 22500 raft_consensus.cc:482] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:27.395159 22500 raft_consensus.cc:504] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.395349 22464 consensus_queue.cc:260] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.395876 21345 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Sending DeleteTablet for 3 replicas of tablet aac61d5d826042e4bcebdcfde8b71a1e
I20260806 02:45:27.396051 22500 leader_election.cc:288] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:27.397737 22464 ts_tablet_manager.cc:1428] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 02:45:27.398479 21345 catalog_manager.cc:2709] Deleting table default.table_c2d5cb4cc8a44eaca15ba851ecc366c8 [id=ccf2db5226644b498abe88bba95d6fef] in response to Hive Metastore notification log event 29
I20260806 02:45:27.399919 21705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea6ade491c6486393aca875767dc1b3" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" is_pre_election: true
W20260806 02:45:27.401846 21436 leader_election.cc:341] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Illegal state: must be running to vote when last-logged opid is not known
I20260806 02:45:27.402698 21681 tablet_service.cc:1514] Processing DeleteTablet for tablet aac61d5d826042e4bcebdcfde8b71a1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:45:27 UTC) from {username='slave'} at 127.0.0.1:38406
I20260806 02:45:27.404014 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea6ade491c6486393aca875767dc1b3" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:27.404495 21888 raft_consensus.cc:2454] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 0.
I20260806 02:45:27.404908 21434 leader_election.cc:302] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:27.405303 22500 raft_consensus.cc:2781] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:27.405441 22500 raft_consensus.cc:482] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:27.405637 22500 raft_consensus.cc:3037] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.405840 21251 catalog_manager.cc:4841] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): tablet d25a9c5a0123446dac00ad895381f96a (table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e]) successfully deleted
I20260806 02:45:27.407843 22444 tablet_replica.cc:330] T aac61d5d826042e4bcebdcfde8b71a1e P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:27.408103 22444 raft_consensus.cc:2229] T aac61d5d826042e4bcebdcfde8b71a1e P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:27.408268 22444 raft_consensus.cc:2258] T aac61d5d826042e4bcebdcfde8b71a1e P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:27.409148 22444 ts_tablet_manager.cc:1905] T aac61d5d826042e4bcebdcfde8b71a1e P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:27.409695 22500 raft_consensus.cc:504] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.410717 21705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea6ade491c6486393aca875767dc1b3" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
W20260806 02:45:27.411367 21436 leader_election.cc:341] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Illegal state: must be running to vote when last-logged opid is not known
I20260806 02:45:27.412127 21863 tablet_service.cc:1514] Processing DeleteTablet for tablet aac61d5d826042e4bcebdcfde8b71a1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:45:27 UTC) from {username='slave'} at 127.0.0.1:56374
I20260806 02:45:27.413554 22445 tablet_replica.cc:330] T aac61d5d826042e4bcebdcfde8b71a1e P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:27.413931 22445 raft_consensus.cc:2229] T aac61d5d826042e4bcebdcfde8b71a1e P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:27.414191 22445 raft_consensus.cc:2258] T aac61d5d826042e4bcebdcfde8b71a1e P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:27.414667 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea6ade491c6486393aca875767dc1b3" candidate_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:27.414875 21888 raft_consensus.cc:3037] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.415073 22445 ts_tablet_manager.cc:1905] T aac61d5d826042e4bcebdcfde8b71a1e P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:27.418457 21888 raft_consensus.cc:2454] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in term 1.
I20260806 02:45:27.419000 21434 leader_election.cc:302] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:27.419337 22466 raft_consensus.cc:2781] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:27.419523 22466 raft_consensus.cc:686] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 LEADER]: Becoming Leader. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Running, Role: LEADER
I20260806 02:45:27.419793 22466 consensus_queue.cc:237] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.421183 22444 ts_tablet_manager.cc:1918] T aac61d5d826042e4bcebdcfde8b71a1e P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:27.421468 22444 log.cc:1196] T aac61d5d826042e4bcebdcfde8b71a1e P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/aac61d5d826042e4bcebdcfde8b71a1e
I20260806 02:45:27.423259 21494 tablet_service.cc:1514] Processing DeleteTablet for tablet aac61d5d826042e4bcebdcfde8b71a1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:45:27 UTC) from {username='slave'} at 127.0.0.1:33118
I20260806 02:45:27.424000 22500 leader_election.cc:288] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 1 election: Requested vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:27.423887 22444 ts_tablet_manager.cc:1939] T aac61d5d826042e4bcebdcfde8b71a1e P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:27.424523 22448 tablet_replica.cc:330] T aac61d5d826042e4bcebdcfde8b71a1e P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:27.424890 22448 raft_consensus.cc:2229] T aac61d5d826042e4bcebdcfde8b71a1e P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:27.425137 22448 raft_consensus.cc:2258] T aac61d5d826042e4bcebdcfde8b71a1e P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:27.425848 22448 ts_tablet_manager.cc:1905] T aac61d5d826042e4bcebdcfde8b71a1e P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:27.426203 21255 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet aac61d5d826042e4bcebdcfde8b71a1e (table default.table_a7715d8e001f48b2899c3c2a56a18a06 [id=21d847eb419e4c0998a01f87c8cd92da]) successfully deleted
I20260806 02:45:27.427397 22463 tablet_bootstrap.cc:492] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:27.427626 22463 ts_tablet_manager.cc:1397] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.084s user 0.008s sys 0.012s
I20260806 02:45:27.428797 22463 raft_consensus.cc:348] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.429108 22463 raft_consensus.cc:374] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.429237 22463 raft_consensus.cc:729] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.429838 22463 consensus_queue.cc:260] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.430721 22463 ts_tablet_manager.cc:1428] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 02:45:27.442600 21681 tablet_service.cc:1467] Processing CreateTablet for tablet 10189388393d47ce900b44cbad3ff425 (DEFAULT_TABLE table=default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:27.443200 21681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10189388393d47ce900b44cbad3ff425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:27.444263 22448 ts_tablet_manager.cc:1918] T aac61d5d826042e4bcebdcfde8b71a1e P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:27.444561 22448 log.cc:1196] T aac61d5d826042e4bcebdcfde8b71a1e P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/aac61d5d826042e4bcebdcfde8b71a1e
I20260806 02:45:27.445116 22448 ts_tablet_manager.cc:1939] T aac61d5d826042e4bcebdcfde8b71a1e P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:27.445660 21266 catalog_manager.cc:5495] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f reported cstate change: term changed from 0 to 1, leader changed from <none> to 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2). New cstate: current_term: 1 leader_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: HEALTHY } } }
I20260806 02:45:27.446216 21494 tablet_service.cc:1467] Processing CreateTablet for tablet 10189388393d47ce900b44cbad3ff425 (DEFAULT_TABLE table=default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:27.446357 21251 catalog_manager.cc:4841] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): tablet aac61d5d826042e4bcebdcfde8b71a1e (table default.table_a7715d8e001f48b2899c3c2a56a18a06 [id=21d847eb419e4c0998a01f87c8cd92da]) successfully deleted
I20260806 02:45:27.446640 21494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10189388393d47ce900b44cbad3ff425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:27.446578 21266 catalog_manager.cc:5495] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f reported cstate change: term changed from 0 to 1, leader changed from <none> to 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2). New cstate: current_term: 1 leader_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260806 02:45:27.446975 21345 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Sending DeleteTablet for 3 replicas of tablet 178955fff8a44eb7b629f09adff0d456
I20260806 02:45:27.447167 21863 tablet_service.cc:1467] Processing CreateTablet for tablet 10189388393d47ce900b44cbad3ff425 (DEFAULT_TABLE table=default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:27.447235 21266 catalog_manager.cc:5587] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655) does not have the latest schema for tablet 62b1e9f4216845739fc757c975eb8c02 (table default.table_97ee2606f8aa47dcb2c67a9ae29dd3fb [id=370d5895c35f4c96878270e71cac2443]). Expected version 2 got 0
I20260806 02:45:27.447682 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10189388393d47ce900b44cbad3ff425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:27.447983 21865 tablet_service.cc:1514] Processing DeleteTablet for tablet 178955fff8a44eb7b629f09adff0d456 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:45:27 UTC) from {username='slave'} at 127.0.0.1:56374
I20260806 02:45:27.449205 21495 tablet_service.cc:1514] Processing DeleteTablet for tablet 178955fff8a44eb7b629f09adff0d456 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:45:27 UTC) from {username='slave'} at 127.0.0.1:33118
F20260806 02:45:27.449831 21345 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 31): 31 DROP_TABLE default.table_1ff877d50df44dccb8be0241b847e8c9
I20260806 02:45:27.450229 21682 tablet_service.cc:1514] Processing DeleteTablet for tablet 178955fff8a44eb7b629f09adff0d456 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:45:27 UTC) from {username='slave'} at 127.0.0.1:38406
*** Check failure stack trace: ***
I20260806 02:45:27.450657 22444 tablet_replica.cc:330] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:27.450872 22444 raft_consensus.cc:2229] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:45:27.451155 22444 pending_rounds.cc:62] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f: Trying to abort 1 pending ops.
I20260806 02:45:27.451218 22444 pending_rounds.cc:68] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315391803930058752 op_type: NO_OP noop_request { }
I20260806 02:45:27.451422 22444 raft_consensus.cc:2866] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260806 02:45:27.451725 22444 raft_consensus.cc:2258] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:27.452996 22444 ts_tablet_manager.cc:1905] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:27.453310 22459 tablet_bootstrap.cc:492] T 10189388393d47ce900b44cbad3ff425 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
@ 0x7fe3978e19ad google::LogMessage::Fail() at ??:0
I20260806 02:45:27.455142 22459 tablet_bootstrap.cc:654] T 10189388393d47ce900b44cbad3ff425 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
@ 0x7fe3978e488a google::LogMessage::SendToLog() at ??:0
@ 0x7fe3978e14de google::LogMessage::Flush() at ??:0
I20260806 02:45:27.457659 22448 tablet_replica.cc:330] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:27.457875 22448 raft_consensus.cc:2229] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:27.458196 22448 raft_consensus.cc:2258] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:27.459311 22448 ts_tablet_manager.cc:1905] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:27.466523 22445 ts_tablet_manager.cc:1918] T aac61d5d826042e4bcebdcfde8b71a1e P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:27.466714 22445 log.cc:1196] T aac61d5d826042e4bcebdcfde8b71a1e P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/aac61d5d826042e4bcebdcfde8b71a1e
I20260806 02:45:27.467124 22445 ts_tablet_manager.cc:1939] T aac61d5d826042e4bcebdcfde8b71a1e P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:27.468098 21251 catalog_manager.cc:4841] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): tablet aac61d5d826042e4bcebdcfde8b71a1e (table default.table_a7715d8e001f48b2899c3c2a56a18a06 [id=21d847eb419e4c0998a01f87c8cd92da]) successfully deleted
I20260806 02:45:27.468183 22464 tablet_bootstrap.cc:492] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:27.468464 22444 ts_tablet_manager.cc:1918] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:27.468782 22444 log.cc:1196] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/178955fff8a44eb7b629f09adff0d456
I20260806 02:45:27.470359 22464 tablet_bootstrap.cc:654] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:27.472967 22445 tablet_replica.cc:330] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:27.473176 22445 raft_consensus.cc:2229] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:27.473380 22445 raft_consensus.cc:2258] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:27.474470 22444 ts_tablet_manager.cc:1939] T 178955fff8a44eb7b629f09adff0d456 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:27.476464 21255 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet 178955fff8a44eb7b629f09adff0d456 (table default.table_c2d5cb4cc8a44eaca15ba851ecc366c8 [id=ccf2db5226644b498abe88bba95d6fef]) successfully deleted
I20260806 02:45:27.476745 22448 ts_tablet_manager.cc:1918] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:27.476923 22448 log.cc:1196] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/178955fff8a44eb7b629f09adff0d456
I20260806 02:45:27.477432 22448 ts_tablet_manager.cc:1939] T 178955fff8a44eb7b629f09adff0d456 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
@ 0x7fe3978e16d1 google::LogMessage::~LogMessage() at ??:0
I20260806 02:45:27.478708 22463 tablet_bootstrap.cc:492] T 10189388393d47ce900b44cbad3ff425 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:27.481266 22463 tablet_bootstrap.cc:654] T 10189388393d47ce900b44cbad3ff425 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:27.483815 21266 catalog_manager.cc:5495] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1). New cstate: current_term: 1 leader_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260806 02:45:27.486325 22445 ts_tablet_manager.cc:1905] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:27.486975 22407 raft_consensus.cc:482] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:27.487149 22407 raft_consensus.cc:504] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.487861 22407 leader_election.cc:288] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:27.489070 21520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca768d858ace4b2b900648ac35f91ecd" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:27.489379 21520 raft_consensus.cc:2454] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:27.489782 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca768d858ace4b2b900648ac35f91ecd" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:27.490074 21888 raft_consensus.cc:2454] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:27.490690 21617 leader_election.cc:302] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:27.491024 22407 raft_consensus.cc:2781] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:27.491144 22407 raft_consensus.cc:482] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:27.491238 22407 raft_consensus.cc:3037] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.494849 22407 raft_consensus.cc:504] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.496531 21520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca768d858ace4b2b900648ac35f91ecd" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:27.496769 22407 leader_election.cc:288] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:27.496956 21520 raft_consensus.cc:3037] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.497985 21266 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46462:
table { table_name: "default.Table_c453f255605e4419859a21f098c7243e" } modify_external_catalogs: true
I20260806 02:45:27.498343 21266 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46462:
table { table_name: "default.Table_c453f255605e4419859a21f098c7243e" } modify_external_catalogs: true
I20260806 02:45:27.504563 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca768d858ace4b2b900648ac35f91ecd" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:27.505023 21888 raft_consensus.cc:3037] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.505450 21251 catalog_manager.cc:4841] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): tablet 178955fff8a44eb7b629f09adff0d456 (table default.table_c2d5cb4cc8a44eaca15ba851ecc366c8 [id=ccf2db5226644b498abe88bba95d6fef]) successfully deleted
I20260806 02:45:27.505895 22464 tablet_bootstrap.cc:492] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:27.506096 22464 ts_tablet_manager.cc:1397] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.038s user 0.009s sys 0.003s
I20260806 02:45:27.506383 21520 raft_consensus.cc:2454] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:27.507102 21617 leader_election.cc:302] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f; no voters:
I20260806 02:45:27.507046 22464 raft_consensus.cc:348] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.507328 22464 raft_consensus.cc:374] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.507416 22464 raft_consensus.cc:729] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.507643 22407 raft_consensus.cc:2781] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:27.507786 22464 consensus_queue.cc:260] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.508780 22407 raft_consensus.cc:686] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Becoming Leader. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Running, Role: LEADER
I20260806 02:45:27.509104 22407 consensus_queue.cc:237] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.509792 22459 tablet_bootstrap.cc:492] T 10189388393d47ce900b44cbad3ff425 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:27.509996 22463 tablet_bootstrap.cc:492] T 10189388393d47ce900b44cbad3ff425 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:27.509969 22459 ts_tablet_manager.cc:1397] T 10189388393d47ce900b44cbad3ff425 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.057s user 0.004s sys 0.049s
I20260806 02:45:27.511036 22459 raft_consensus.cc:348] T 10189388393d47ce900b44cbad3ff425 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.511384 22459 raft_consensus.cc:374] T 10189388393d47ce900b44cbad3ff425 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.511616 22459 raft_consensus.cc:729] T 10189388393d47ce900b44cbad3ff425 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.512152 22459 consensus_queue.cc:260] T 10189388393d47ce900b44cbad3ff425 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.512671 22463 ts_tablet_manager.cc:1397] T 10189388393d47ce900b44cbad3ff425 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.034s user 0.008s sys 0.002s
I20260806 02:45:27.513686 22463 raft_consensus.cc:348] T 10189388393d47ce900b44cbad3ff425 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.513995 22463 raft_consensus.cc:374] T 10189388393d47ce900b44cbad3ff425 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:27.514087 22463 raft_consensus.cc:729] T 10189388393d47ce900b44cbad3ff425 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:27.514348 22463 consensus_queue.cc:260] T 10189388393d47ce900b44cbad3ff425 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:27.515275 22463 ts_tablet_manager.cc:1428] T 10189388393d47ce900b44cbad3ff425 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 02:45:27.515650 22459 ts_tablet_manager.cc:1428] T 10189388393d47ce900b44cbad3ff425 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260806 02:45:27.519492 21888 raft_consensus.cc:2454] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
@ 0x7fe3ad47fba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260806 02:45:27.523676 22445 ts_tablet_manager.cc:1918] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:27.523831 22445 log.cc:1196] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/178955fff8a44eb7b629f09adff0d456
I20260806 02:45:27.524326 22445 ts_tablet_manager.cc:1939] T 178955fff8a44eb7b629f09adff0d456 P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:27.525679 22464 ts_tablet_manager.cc:1428] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.019s user 0.000s sys 0.003s
I20260806 02:45:27.527694 21251 catalog_manager.cc:4841] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): tablet 178955fff8a44eb7b629f09adff0d456 (table default.table_c2d5cb4cc8a44eaca15ba851ecc366c8 [id=ccf2db5226644b498abe88bba95d6fef]) successfully deleted
@ 0x7fe3ad47e901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7fe3ad48566c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260806 02:45:27.540858 21888 raft_consensus.cc:1261] T 62b1e9f4216845739fc757c975eb8c02 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Refusing update from remote peer 88dc3ab284a64961b1fbdb5c53d1267f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:45:27.541092 21520 raft_consensus.cc:1261] T 62b1e9f4216845739fc757c975eb8c02 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Refusing update from remote peer 88dc3ab284a64961b1fbdb5c53d1267f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:45:27.541664 22407 consensus_queue.cc:1035] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
@ 0x7fe3ad4854e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260806 02:45:27.542065 22497 consensus_queue.cc:1035] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe39a50beaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe39d7e56db start_thread at ??:0
@ 0x7fe39677b61f clone at ??:0
I20260806 02:45:27.569650 22543 tablet.cc:1722] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260806 02:45:27.570940 22544 tablet.cc:1722] T 62b1e9f4216845739fc757c975eb8c02 P 8ebd96b903b944f3b06851324b737bd5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260806 02:45:27.615083 22554 tablet.cc:1722] T 62b1e9f4216845739fc757c975eb8c02 P 48b0c1e2106b4d9b941929ddc6d97dd5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260806 02:45:27.621932 22543 tablet.cc:1717] T 62b1e9f4216845739fc757c975eb8c02 P 88dc3ab284a64961b1fbdb5c53d1267f: Skipping requested alter to schema version 2, tablet already version 2
I20260806 02:45:27.628419 22544 tablet.cc:1717] T 62b1e9f4216845739fc757c975eb8c02 P 8ebd96b903b944f3b06851324b737bd5: Skipping requested alter to schema version 2, tablet already version 2
I20260806 02:45:27.628396 22554 tablet.cc:1717] T 62b1e9f4216845739fc757c975eb8c02 P 48b0c1e2106b4d9b941929ddc6d97dd5: Skipping requested alter to schema version 2, tablet already version 2
I20260806 02:45:27.698777 20720 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu pid 21228
I20260806 02:45:27.712266 22489 consensus_queue.cc:1035] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:27.721776 22489 consensus_queue.cc:1035] T a6e1420c19fc44b3901f653f137ec6c1 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:27.787740 22466 consensus_queue.cc:1035] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:27.793834 22501 consensus_queue.cc:1035] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 02:45:27.853293 21978 connection.cc:535] client connection to 127.20.60.60:42309 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 02:45:27.853703 21045 master_runner.cc:186] Error getting master registration for 127.20.60.60:42309: Network error: recv got EOF from 127.20.60.60:42309 (error 108),
I20260806 02:45:27.853928 21045 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:45:27.854333 21978 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.20.60.60:42309)
W20260806 02:45:27.854652 21978 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.20.60.60:42309)
W20260806 02:45:27.855410 20727 connection.cc:535] client connection to 127.20.60.60:42309 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:45:27.855629 21978 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.20.60.60:42309)
W20260806 02:45:27.856387 20720 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu pid 21228: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 02:45:27.856555 20720 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu with pid 21228
W20260806 02:45:27.857512 21620 connection.cc:535] server connection from 127.0.0.1:38406 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:45:27.859755 21978 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.20.60.60:42309)
I20260806 02:45:27.852795 22091 master_runner.cc:186] Error getting master registration for 127.20.60.60:42309: Network error: recv got EOF from 127.20.60.60:42309 (error 108),
I20260806 02:45:27.864814 22091 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:45:27.895339 22548 consensus_queue.cc:1035] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:27.900720 22550 consensus_queue.cc:1035] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:27.932024 22550 raft_consensus.cc:482] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:27.932271 22550 raft_consensus.cc:504] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:27.932972 22550 leader_election.cc:288] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:27.933662 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea6ade491c6486393aca875767dc1b3" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:27.933686 21519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bea6ade491c6486393aca875767dc1b3" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:27.933970 21888 raft_consensus.cc:2379] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88dc3ab284a64961b1fbdb5c53d1267f in current term 1: Already voted for candidate 48b0c1e2106b4d9b941929ddc6d97dd5 in this term.
I20260806 02:45:27.934543 21617 leader_election.cc:302] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:45:27.934877 22550 raft_consensus.cc:3037] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:27.936985 22550 raft_consensus.cc:2726] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:45:27.994210 22574 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 894be089543b4aa29b3edfcc5f3085e9)
I20260806 02:45:27.994452 22574 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:27.994570 22576 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 894be089543b4aa29b3edfcc5f3085e9)
I20260806 02:45:27.994812 22576 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:27.995319 22574 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
I20260806 02:45:27.995849 22303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a225e002d607451a94b3686f10aeea9a" candidate_term: 5 candidate_status { last_received { term: 4 index: 97 } } ignore_live_leader: false dest_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" is_pre_election: true
I20260806 02:45:27.996102 22303 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a225e002d607451a94b3686f10aeea9a in term 4.
W20260806 02:45:27.996362 21068 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111) [suppressed 4 similar messages]
I20260806 02:45:27.996585 21064 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22; no voters:
I20260806 02:45:27.996914 22574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260806 02:45:27.997072 22574 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 894be089543b4aa29b3edfcc5f3085e9)
I20260806 02:45:27.997267 22574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 4 FOLLOWER]: Advancing to term 5
I20260806 02:45:27.997792 21095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" candidate_term: 5 candidate_status { last_received { term: 4 index: 97 } } ignore_live_leader: false dest_uuid: "a225e002d607451a94b3686f10aeea9a" is_pre_election: true
W20260806 02:45:27.997993 21068 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:27.998355 22576 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:27.999253 22243 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:28.000418 22574 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:28.000806 21095 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bfdbf4ef60014ea98bbcf9b6a9033f22 in current term 5: Already voted for candidate a225e002d607451a94b3686f10aeea9a in this term.
W20260806 02:45:28.000864 22243 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:28.001083 22574 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 5 election: Requested vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
I20260806 02:45:28.001538 22303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a225e002d607451a94b3686f10aeea9a" candidate_term: 5 candidate_status { last_received { term: 4 index: 97 } } ignore_live_leader: false dest_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22"
I20260806 02:45:28.001777 22303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 4 FOLLOWER]: Advancing to term 5
I20260806 02:45:28.001788 22242 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a
W20260806 02:45:28.002815 21068 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:28.003947 22303 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a225e002d607451a94b3686f10aeea9a in term 5.
I20260806 02:45:28.004132 22576 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260806 02:45:28.004348 21064 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: 894be089543b4aa29b3edfcc5f3085e9
I20260806 02:45:28.004652 22574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 5 FOLLOWER]: Leader election won for term 5
I20260806 02:45:28.005110 22574 raft_consensus.cc:686] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 5 LEADER]: Becoming Leader. State: Replica: a225e002d607451a94b3686f10aeea9a, State: Running, Role: LEADER
I20260806 02:45:28.005439 22574 consensus_queue.cc:237] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 97, Committed index: 97, Last appended: 4.97, Last appended by leader: 97, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:28.007411 22582 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a225e002d607451a94b3686f10aeea9a. Latest consensus state: current_term: 5 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:28.007776 22582 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:28.008450 22583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:45:28.011168 22583 catalog_manager.cc:669] Loaded metadata for table default.table_e41f56f2e4a1439ca6b8f2bca460a670 [id=1bbd1184e7754c5bb9ffdf82791c73fb]
I20260806 02:45:28.011919 22583 catalog_manager.cc:669] Loaded metadata for table default.table_97ee2606f8aa47dcb2c67a9ae29dd3fb [id=370d5895c35f4c96878270e71cac2443]
I20260806 02:45:28.012244 22583 catalog_manager.cc:669] Loaded metadata for table default.table_ea1a38afd92a4b15bfabb80e58f8c644 [id=383ba93bcba140c1b1bd2b36f8720b86]
I20260806 02:45:28.012619 22583 catalog_manager.cc:669] Loaded metadata for table default.table_d7249617a95049389b90db3bd551ace4 [id=4179f72ea45043e39ddce8e93c3429e9]
I20260806 02:45:28.012934 22583 catalog_manager.cc:669] Loaded metadata for table default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e]
I20260806 02:45:28.013288 22583 catalog_manager.cc:669] Loaded metadata for table default.table_0febb4673dff4e20ae8124efba9627c8 [id=5cb271f99441401cad4fd2e40e943a6e]
I20260806 02:45:28.013628 22583 catalog_manager.cc:669] Loaded metadata for table default.table_21221d53bf9f4c46a5ffe671a1ffe424 [id=610932bdd3f94b4e886d586c28ff6249]
I20260806 02:45:28.013978 22583 catalog_manager.cc:669] Loaded metadata for table default.table_e8df6dde49954df7b57567fc0cd12872 [id=a0cffb03ae7641f8848b8d2aab0fca8b]
I20260806 02:45:28.014329 22583 catalog_manager.cc:669] Loaded metadata for table default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26]
I20260806 02:45:28.014681 22583 catalog_manager.cc:669] Loaded metadata for table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]
I20260806 02:45:28.014998 22583 catalog_manager.cc:669] Loaded metadata for table default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c]
I20260806 02:45:28.015329 22583 catalog_manager.cc:669] Loaded metadata for table default.table_00c06d2b7d6d40658a4390f4ee3bbab5 [id=c6a05485b0a54967ba8904aa969edd3f]
I20260806 02:45:28.015849 22583 catalog_manager.cc:669] Loaded metadata for table default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704]
I20260806 02:45:28.016220 22583 catalog_manager.cc:669] Loaded metadata for table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e]
I20260806 02:45:28.016554 22583 catalog_manager.cc:669] Loaded metadata for table default.table_c5be13d5ad2c4a479d2cc9074d14038f [id=f6939f0d45ae4f358d206a8cf2fe5db0]
I20260806 02:45:28.018780 22583 tablet_loader.cc:96] loaded metadata for tablet 043839a5eca143d5882f40faaab39692 (table default.table_21221d53bf9f4c46a5ffe671a1ffe424 [id=610932bdd3f94b4e886d586c28ff6249])
I20260806 02:45:28.019327 22583 tablet_loader.cc:96] loaded metadata for tablet 10189388393d47ce900b44cbad3ff425 (table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e])
I20260806 02:45:28.019809 22583 tablet_loader.cc:96] loaded metadata for tablet 237439c720344cfd97bd04bf8368a375 (table default.table_ea1a38afd92a4b15bfabb80e58f8c644 [id=383ba93bcba140c1b1bd2b36f8720b86])
I20260806 02:45:28.020133 22583 tablet_loader.cc:96] loaded metadata for tablet 2d8219e0438e46f783037537e687f945 (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022])
I20260806 02:45:28.020442 22583 tablet_loader.cc:96] loaded metadata for tablet 3196f6ee70d34d298b7db56d9759fab7 (table default.table_00c06d2b7d6d40658a4390f4ee3bbab5 [id=c6a05485b0a54967ba8904aa969edd3f])
I20260806 02:45:28.020874 22583 tablet_loader.cc:96] loaded metadata for tablet 3d44c58ba1f044b38f10da10cec410e3 (table default.table_e41f56f2e4a1439ca6b8f2bca460a670 [id=1bbd1184e7754c5bb9ffdf82791c73fb])
I20260806 02:45:28.021441 22583 tablet_loader.cc:96] loaded metadata for tablet 6061f59aedec4e0a9b96239ebe6fa0df (table default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e])
I20260806 02:45:28.021890 22583 tablet_loader.cc:96] loaded metadata for tablet 62b1e9f4216845739fc757c975eb8c02 (table default.table_97ee2606f8aa47dcb2c67a9ae29dd3fb [id=370d5895c35f4c96878270e71cac2443])
I20260806 02:45:28.022528 22583 tablet_loader.cc:96] loaded metadata for tablet 81dd98ae43b7429992d48eeaa9f17a70 (table default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26])
I20260806 02:45:28.022996 22583 tablet_loader.cc:96] loaded metadata for tablet a6e1420c19fc44b3901f653f137ec6c1 (table default.table_e8df6dde49954df7b57567fc0cd12872 [id=a0cffb03ae7641f8848b8d2aab0fca8b])
I20260806 02:45:28.023612 22583 tablet_loader.cc:96] loaded metadata for tablet ba32fbee948d4adf96164d36673a1db6 (table default.table_0febb4673dff4e20ae8124efba9627c8 [id=5cb271f99441401cad4fd2e40e943a6e])
I20260806 02:45:28.023924 22583 tablet_loader.cc:96] loaded metadata for tablet bea6ade491c6486393aca875767dc1b3 (table default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704])
I20260806 02:45:28.024351 22583 tablet_loader.cc:96] loaded metadata for tablet ca768d858ace4b2b900648ac35f91ecd (table default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c])
I20260806 02:45:28.025290 22583 tablet_loader.cc:96] loaded metadata for tablet f9fd42a920b24117a4d687fec13ad17b (table default.table_d7249617a95049389b90db3bd551ace4 [id=4179f72ea45043e39ddce8e93c3429e9])
I20260806 02:45:28.025663 22583 tablet_loader.cc:96] loaded metadata for tablet fdc39b7dea2c43cebc730cffacd38d10 (table default.table_c5be13d5ad2c4a479d2cc9074d14038f [id=f6939f0d45ae4f358d206a8cf2fe5db0])
I20260806 02:45:28.026600 22583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:45:28.028182 22583 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:28.028283 22583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:45:28.030078 22583 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:45:28.031390 22583 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Loaded TSK: 0
I20260806 02:45:28.031869 22583 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:45:28.033097 22583 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260806 02:45:28.034441 22583 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: 29
I20260806 02:45:28.048231 21081 master_service.cc:946] ReplaceTablet: received request to replace tablet ca768d858ace4b2b900648ac35f91ecd from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:28.048692 21082 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_468b3a35d3fb4572bc7b89d436d4e319"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:28.049503 21083 master_service.cc:946] ReplaceTablet: received request to replace tablet 81dd98ae43b7429992d48eeaa9f17a70 from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:28.049834 21079 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_8a4c223baf6149efb9e759a304f00edd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:28.049503 21077 master_service.cc:946] ReplaceTablet: received request to replace tablet 2d8219e0438e46f783037537e687f945 from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:28.051278 21080 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_8ee8dd77f08a4c9d9e3d4f345a9a6bbf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:28.051478 21084 master_service.cc:946] ReplaceTablet: received request to replace tablet 2d8219e0438e46f783037537e687f945 from {username='slave'} at 127.0.0.1:58094
I20260806 02:45:28.052050 21076 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_cac77bd182314ee2a2068374ddeca8d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:45:28.056514 21068 consensus_peers.cc:487] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a -> Peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Couldn't send request to peer 894be089543b4aa29b3edfcc5f3085e9. Status: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 02:45:28.057039 21074 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_3945346dd2514eddba4dfc965eb5d995"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:28.057622 21073 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_1ff877d50df44dccb8be0241b847e8c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260806 02:45:28.057942 21391 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 31): 31 DROP_TABLE default.table_1ff877d50df44dccb8be0241b847e8c9
I20260806 02:45:28.058089 22303 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 5 FOLLOWER]: Refusing update from remote peer a225e002d607451a94b3686f10aeea9a: Log matching property violated. Preceding OpId in replica: term: 4 index: 97. Preceding OpId from leader: term: 5 index: 100. (index mismatch)
*** Check failure stack trace: ***
I20260806 02:45:28.058820 22586 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 98, Last known committed idx: 97, Time since last communication: 0.000s
@ 0x7f1a1c3b09ad google::LogMessage::Fail() at ??:0
@ 0x7f1a1c3b388a google::LogMessage::SendToLog() at ??:0
@ 0x7f1a1c3b04de google::LogMessage::Flush() at ??:0
@ 0x7f1a1c3b06d1 google::LogMessage::~LogMessage() at ??:0
I20260806 02:45:28.066599 22576 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a225e002d607451a94b3686f10aeea9a. Latest consensus state: current_term: 5 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:28.066906 22576 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f1a31f4eba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260806 02:45:28.076537 21081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Sending DeleteTablet for 3 replicas of tablet ca768d858ace4b2b900648ac35f91ecd
I20260806 02:45:28.077142 21081 catalog_manager.cc:6565] ReplaceTablet: tablet ca768d858ace4b2b900648ac35f91ecd deleted and replaced by tablet 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:45:28.078408 21681 tablet_service.cc:1514] Processing DeleteTablet for tablet ca768d858ace4b2b900648ac35f91ecd with delete_type TABLET_DATA_DELETED (replaced by tablet 6cc50b1bfe414464b1863c4e26f295fb) from {username='slave'} at 127.0.0.1:38404
I20260806 02:45:28.078750 21494 tablet_service.cc:1514] Processing DeleteTablet for tablet ca768d858ace4b2b900648ac35f91ecd with delete_type TABLET_DATA_DELETED (replaced by tablet 6cc50b1bfe414464b1863c4e26f295fb) from {username='slave'} at 127.0.0.1:33104
I20260806 02:45:28.078877 21863 tablet_service.cc:1514] Processing DeleteTablet for tablet ca768d858ace4b2b900648ac35f91ecd with delete_type TABLET_DATA_DELETED (replaced by tablet 6cc50b1bfe414464b1863c4e26f295fb) from {username='slave'} at 127.0.0.1:56372
I20260806 02:45:28.086844 21083 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Sending DeleteTablet for 3 replicas of tablet 81dd98ae43b7429992d48eeaa9f17a70
I20260806 02:45:28.087352 21083 catalog_manager.cc:6565] ReplaceTablet: tablet 81dd98ae43b7429992d48eeaa9f17a70 deleted and replaced by tablet 293b93e6322842aaaa786570f2810548
I20260806 02:45:28.087918 21083 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_5ed544635d9c480cb8c61f25b7f0f84f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:28.088027 21495 tablet_service.cc:1514] Processing DeleteTablet for tablet 81dd98ae43b7429992d48eeaa9f17a70 with delete_type TABLET_DATA_DELETED (replaced by tablet 293b93e6322842aaaa786570f2810548) from {username='slave'} at 127.0.0.1:33104
I20260806 02:45:28.089324 21682 tablet_service.cc:1514] Processing DeleteTablet for tablet 81dd98ae43b7429992d48eeaa9f17a70 with delete_type TABLET_DATA_DELETED (replaced by tablet 293b93e6322842aaaa786570f2810548) from {username='slave'} at 127.0.0.1:38404
I20260806 02:45:28.089900 21865 tablet_service.cc:1514] Processing DeleteTablet for tablet 81dd98ae43b7429992d48eeaa9f17a70 with delete_type TABLET_DATA_DELETED (replaced by tablet 293b93e6322842aaaa786570f2810548) from {username='slave'} at 127.0.0.1:56372
I20260806 02:45:28.091331 21081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58094:
table { table_name: "default.Table_0febb4673dff4e20ae8124efba9627c8" } modify_external_catalogs: true
I20260806 02:45:28.091557 21081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58094:
table { table_name: "default.Table_0febb4673dff4e20ae8124efba9627c8" } modify_external_catalogs: true
I20260806 02:45:28.091879 22574 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:28.092142 22574 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:28.099423 22576 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:28.099771 22576 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:28.102048 21077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Sending DeleteTablet for 3 replicas of tablet 2d8219e0438e46f783037537e687f945
I20260806 02:45:28.102618 21077 catalog_manager.cc:6565] ReplaceTablet: tablet 2d8219e0438e46f783037537e687f945 deleted and replaced by tablet 50c46742a5a44d54aba65eec2f36aa09
I20260806 02:45:28.103061 21865 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d8219e0438e46f783037537e687f945 with delete_type TABLET_DATA_DELETED (replaced by tablet 50c46742a5a44d54aba65eec2f36aa09) from {username='slave'} at 127.0.0.1:56372
I20260806 02:45:28.103084 21077 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58094:
name: "default.Table_6d994a6895c3485cb404565731c2e7fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:28.103430 21682 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d8219e0438e46f783037537e687f945 with delete_type TABLET_DATA_DELETED (replaced by tablet 50c46742a5a44d54aba65eec2f36aa09) from {username='slave'} at 127.0.0.1:38404
I20260806 02:45:28.103844 21084 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58094:
table { table_name: "default.Table_c453f255605e4419859a21f098c7243e" } modify_external_catalogs: true
I20260806 02:45:28.104076 21084 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58094:
table { table_name: "default.Table_c453f255605e4419859a21f098c7243e" } modify_external_catalogs: true
I20260806 02:45:28.104664 21495 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d8219e0438e46f783037537e687f945 with delete_type TABLET_DATA_DELETED (replaced by tablet 50c46742a5a44d54aba65eec2f36aa09) from {username='slave'} at 127.0.0.1:33104
I20260806 02:45:28.107607 21865 tablet_service.cc:1467] Processing CreateTablet for tablet 293b93e6322842aaaa786570f2810548 (DEFAULT_TABLE table=default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:28.108147 21865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 293b93e6322842aaaa786570f2810548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:28.110316 21495 tablet_service.cc:1467] Processing CreateTablet for tablet 293b93e6322842aaaa786570f2810548 (DEFAULT_TABLE table=default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:28.110435 21683 tablet_service.cc:1467] Processing CreateTablet for tablet 6cc50b1bfe414464b1863c4e26f295fb (DEFAULT_TABLE table=default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:28.110749 21495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 293b93e6322842aaaa786570f2810548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:28.110982 21683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cc50b1bfe414464b1863c4e26f295fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:28.113679 21682 tablet_service.cc:1467] Processing CreateTablet for tablet 293b93e6322842aaaa786570f2810548 (DEFAULT_TABLE table=default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:28.114156 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 293b93e6322842aaaa786570f2810548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:28.114919 22600 tablet_replica.cc:330] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:28.115139 22600 raft_consensus.cc:2229] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:28.115394 22600 raft_consensus.cc:2258] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:28.116004 22600 ts_tablet_manager.cc:1905] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:28.117681 21498 tablet_service.cc:1467] Processing CreateTablet for tablet 6cc50b1bfe414464b1863c4e26f295fb (DEFAULT_TABLE table=default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:28.117843 21862 tablet_service.cc:1467] Processing CreateTablet for tablet 6cc50b1bfe414464b1863c4e26f295fb (DEFAULT_TABLE table=default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:28.118142 21498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cc50b1bfe414464b1863c4e26f295fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:28.118376 21862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cc50b1bfe414464b1863c4e26f295fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:28.118420 22596 tablet_replica.cc:330] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:28.118661 22596 raft_consensus.cc:2229] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:45:28.118939 22596 pending_rounds.cc:62] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f: Trying to abort 1 pending ops.
I20260806 02:45:28.118997 22596 pending_rounds.cc:68] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315391805481107456 op_type: NO_OP noop_request { }
I20260806 02:45:28.119171 22596 raft_consensus.cc:2866] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260806 02:45:28.119273 22596 raft_consensus.cc:2258] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:28.121271 22599 tablet_replica.cc:330] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:28.122120 22599 raft_consensus.cc:2229] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:28.122350 22599 raft_consensus.cc:2258] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:28.122959 22599 ts_tablet_manager.cc:1905] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:28.125582 22596 ts_tablet_manager.cc:1905] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:28.133536 21498 tablet_service.cc:1467] Processing CreateTablet for tablet 50c46742a5a44d54aba65eec2f36aa09 (DEFAULT_TABLE table=default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:28.133800 21863 tablet_service.cc:1467] Processing CreateTablet for tablet 50c46742a5a44d54aba65eec2f36aa09 (DEFAULT_TABLE table=default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:28.133968 21498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50c46742a5a44d54aba65eec2f36aa09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:28.134202 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50c46742a5a44d54aba65eec2f36aa09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:28.137068 22602 tablet_bootstrap.cc:492] T 293b93e6322842aaaa786570f2810548 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:28.137727 21681 tablet_service.cc:1467] Processing CreateTablet for tablet 50c46742a5a44d54aba65eec2f36aa09 (DEFAULT_TABLE table=default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:28.138135 21681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50c46742a5a44d54aba65eec2f36aa09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:28.138952 22602 tablet_bootstrap.cc:654] T 293b93e6322842aaaa786570f2810548 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:28.141177 22601 tablet_bootstrap.cc:492] T 293b93e6322842aaaa786570f2810548 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
@ 0x7f1a31f4d901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260806 02:45:28.141762 22604 tablet_bootstrap.cc:492] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:28.141625 22550 consensus_queue.cc:1035] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:28.143750 22604 tablet_bootstrap.cc:654] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:28.145510 22412 consensus_queue.cc:1035] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
@ 0x7f1a31f5466c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260806 02:45:28.152611 22601 tablet_bootstrap.cc:654] T 293b93e6322842aaaa786570f2810548 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
@ 0x7f1a31f544e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260806 02:45:28.159050 22602 tablet_bootstrap.cc:492] T 293b93e6322842aaaa786570f2810548 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:28.159222 22602 ts_tablet_manager.cc:1397] T 293b93e6322842aaaa786570f2810548 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.000s
I20260806 02:45:28.160404 22497 consensus_queue.cc:1035] T fdc39b7dea2c43cebc730cffacd38d10 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:28.160100 22602 raft_consensus.cc:348] T 293b93e6322842aaaa786570f2810548 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.163154 22602 raft_consensus.cc:374] T 293b93e6322842aaaa786570f2810548 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:28.163293 22602 raft_consensus.cc:729] T 293b93e6322842aaaa786570f2810548 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:28.164351 22602 consensus_queue.cc:260] T 293b93e6322842aaaa786570f2810548 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.165182 22602 ts_tablet_manager.cc:1428] T 293b93e6322842aaaa786570f2810548 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260806 02:45:28.165580 22602 tablet_bootstrap.cc:492] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:28.167574 22601 tablet_bootstrap.cc:492] T 293b93e6322842aaaa786570f2810548 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:28.167729 22601 ts_tablet_manager.cc:1397] T 293b93e6322842aaaa786570f2810548 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.006s
I20260806 02:45:28.167907 22604 tablet_bootstrap.cc:492] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:28.168054 22604 ts_tablet_manager.cc:1397] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.006s
I20260806 02:45:28.168566 22601 raft_consensus.cc:348] T 293b93e6322842aaaa786570f2810548 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.168820 22601 raft_consensus.cc:374] T 293b93e6322842aaaa786570f2810548 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:28.168916 22601 raft_consensus.cc:729] T 293b93e6322842aaaa786570f2810548 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:28.168973 22604 raft_consensus.cc:348] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.169243 22604 raft_consensus.cc:374] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:28.169385 22604 raft_consensus.cc:729] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:28.169641 22604 consensus_queue.cc:260] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260806 02:45:28.170328 22604 ts_tablet_manager.cc:1428] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 02:45:28.170750 22604 tablet_bootstrap.cc:492] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:28.172766 22604 tablet_bootstrap.cc:654] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:28.173606 22601 consensus_queue.cc:260] T 293b93e6322842aaaa786570f2810548 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.178905 22602 tablet_bootstrap.cc:654] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:28.180380 22489 consensus_queue.cc:1035] T f9fd42a920b24117a4d687fec13ad17b P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:28.182739 22500 consensus_queue.cc:1035] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260806 02:45:28.183262 22601 ts_tablet_manager.cc:1428] T 293b93e6322842aaaa786570f2810548 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.012s user 0.001s sys 0.002s
I20260806 02:45:28.183679 22601 tablet_bootstrap.cc:492] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:28.185534 22604 tablet_bootstrap.cc:492] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:28.185706 22604 ts_tablet_manager.cc:1397] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.002s
I20260806 02:45:28.186717 22604 raft_consensus.cc:348] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.186978 22604 raft_consensus.cc:374] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:28.187065 22604 raft_consensus.cc:729] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:28.187346 22604 consensus_queue.cc:260] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
@ 0x7f1a1efdaeaf kudu::Thread::SuperviseThread() at ??:0
I20260806 02:45:28.187804 22601 tablet_bootstrap.cc:654] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:28.187928 22604 ts_tablet_manager.cc:1428] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 02:45:28.190155 22601 tablet_bootstrap.cc:492] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:28.190330 22601 ts_tablet_manager.cc:1397] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.002s
I20260806 02:45:28.191538 22601 raft_consensus.cc:348] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.191780 22601 raft_consensus.cc:374] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:28.191867 22601 raft_consensus.cc:729] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:28.192109 22601 consensus_queue.cc:260] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.192389 22500 consensus_queue.cc:1035] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:28.192739 22601 ts_tablet_manager.cc:1428] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260806 02:45:28.193081 22601 tablet_bootstrap.cc:492] T 50c46742a5a44d54aba65eec2f36aa09 P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:28.195168 22601 tablet_bootstrap.cc:654] T 50c46742a5a44d54aba65eec2f36aa09 P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:28.204504 22604 tablet_bootstrap.cc:492] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:28.206840 22604 tablet_bootstrap.cc:654] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:28.207357 22596 ts_tablet_manager.cc:1918] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:28.207557 22596 log.cc:1196] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/ca768d858ace4b2b900648ac35f91ecd
I20260806 02:45:28.208055 22596 ts_tablet_manager.cc:1939] T ca768d858ace4b2b900648ac35f91ecd P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:28.209241 22596 tablet_replica.cc:330] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:28.209408 21068 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet ca768d858ace4b2b900648ac35f91ecd (table default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c]) successfully deleted
I20260806 02:45:28.209849 22602 tablet_bootstrap.cc:492] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:28.210050 22602 ts_tablet_manager.cc:1397] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.045s user 0.003s sys 0.002s
I20260806 02:45:28.210951 22602 raft_consensus.cc:348] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.211274 22600 ts_tablet_manager.cc:1918] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:28.211401 22600 log.cc:1196] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/ca768d858ace4b2b900648ac35f91ecd
I20260806 02:45:28.211649 22602 raft_consensus.cc:374] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:28.211745 22602 raft_consensus.cc:729] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:28.211859 22600 ts_tablet_manager.cc:1939] T ca768d858ace4b2b900648ac35f91ecd P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:28.211997 22602 consensus_queue.cc:260] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.212595 22602 ts_tablet_manager.cc:1428] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 02:45:28.212898 22602 tablet_bootstrap.cc:492] T 50c46742a5a44d54aba65eec2f36aa09 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
@ 0x7f1a222b46db start_thread at ??:0
I20260806 02:45:28.215260 22602 tablet_bootstrap.cc:654] T 50c46742a5a44d54aba65eec2f36aa09 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:28.216519 22596 raft_consensus.cc:2229] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:28.216740 22596 raft_consensus.cc:2258] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:28.217976 22601 tablet_bootstrap.cc:492] T 50c46742a5a44d54aba65eec2f36aa09 P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:28.218127 22601 ts_tablet_manager.cc:1397] T 50c46742a5a44d54aba65eec2f36aa09 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.007s
I20260806 02:45:28.218281 22596 ts_tablet_manager.cc:1905] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:28.218881 22601 raft_consensus.cc:348] T 50c46742a5a44d54aba65eec2f36aa09 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.219164 22601 raft_consensus.cc:374] T 50c46742a5a44d54aba65eec2f36aa09 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:28.219246 22601 raft_consensus.cc:729] T 50c46742a5a44d54aba65eec2f36aa09 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:28.219513 22601 consensus_queue.cc:260] T 50c46742a5a44d54aba65eec2f36aa09 P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.220201 22601 ts_tablet_manager.cc:1428] T 50c46742a5a44d54aba65eec2f36aa09 P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 02:45:28.222635 21064 catalog_manager.cc:4841] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): tablet ca768d858ace4b2b900648ac35f91ecd (table default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c]) successfully deleted
I20260806 02:45:28.223140 22600 tablet_replica.cc:330] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:28.223362 22600 raft_consensus.cc:2229] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:28.223524 22600 raft_consensus.cc:2258] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:28.224386 22599 ts_tablet_manager.cc:1918] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 02:45:28.224525 22599 log.cc:1196] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/ca768d858ace4b2b900648ac35f91ecd
I20260806 02:45:28.224912 22599 ts_tablet_manager.cc:1939] T ca768d858ace4b2b900648ac35f91ecd P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:28.225937 21064 catalog_manager.cc:4841] TS 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): tablet ca768d858ace4b2b900648ac35f91ecd (table default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c]) successfully deleted
I20260806 02:45:28.228667 22604 tablet_bootstrap.cc:492] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:28.228827 22604 ts_tablet_manager.cc:1397] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.005s
I20260806 02:45:28.228349 22600 ts_tablet_manager.cc:1905] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:28.229280 22599 tablet_replica.cc:330] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:28.230325 22604 raft_consensus.cc:348] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.231174 22599 raft_consensus.cc:2229] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:45:28.231246 22604 raft_consensus.cc:374] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:28.231354 22604 raft_consensus.cc:729] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:28.231520 22599 raft_consensus.cc:2258] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:28.231611 22604 consensus_queue.cc:260] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.234032 22604 ts_tablet_manager.cc:1428] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20260806 02:45:28.234959 22599 ts_tablet_manager.cc:1905] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f1a1b24a61f clone at ??:0
I20260806 02:45:28.237797 22489 raft_consensus.cc:482] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:28.237938 22489 raft_consensus.cc:504] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:28.238610 22489 leader_election.cc:288] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:28.239259 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10189388393d47ce900b44cbad3ff425" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" is_pre_election: true
I20260806 02:45:28.239472 21710 raft_consensus.cc:2454] T 10189388393d47ce900b44cbad3ff425 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:28.239912 21806 leader_election.cc:302] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:28.239969 21519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10189388393d47ce900b44cbad3ff425" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:28.240211 22489 raft_consensus.cc:2781] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:28.240247 21519 raft_consensus.cc:2454] T 10189388393d47ce900b44cbad3ff425 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:28.240321 22489 raft_consensus.cc:482] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:28.240422 22489 raft_consensus.cc:3037] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:28.242188 22599 ts_tablet_manager.cc:1918] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:28.242322 22599 log.cc:1196] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/81dd98ae43b7429992d48eeaa9f17a70
I20260806 02:45:28.242746 22599 ts_tablet_manager.cc:1939] T 81dd98ae43b7429992d48eeaa9f17a70 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:28.243207 22602 tablet_bootstrap.cc:492] T 50c46742a5a44d54aba65eec2f36aa09 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:28.243350 22602 ts_tablet_manager.cc:1397] T 50c46742a5a44d54aba65eec2f36aa09 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.031s user 0.005s sys 0.003s
I20260806 02:45:28.243789 22599 tablet_replica.cc:330] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:28.243989 22599 raft_consensus.cc:2229] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:28.244207 22599 raft_consensus.cc:2258] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:28.244201 22602 raft_consensus.cc:348] T 50c46742a5a44d54aba65eec2f36aa09 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.244448 22602 raft_consensus.cc:374] T 50c46742a5a44d54aba65eec2f36aa09 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:28.244554 22602 raft_consensus.cc:729] T 50c46742a5a44d54aba65eec2f36aa09 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:28.244840 22599 ts_tablet_manager.cc:1905] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:28.244772 22489 raft_consensus.cc:504] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:28.244833 22602 consensus_queue.cc:260] T 50c46742a5a44d54aba65eec2f36aa09 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.245880 21519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10189388393d47ce900b44cbad3ff425" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:28.246127 21519 raft_consensus.cc:3037] T 10189388393d47ce900b44cbad3ff425 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:28.247291 22602 ts_tablet_manager.cc:1428] T 50c46742a5a44d54aba65eec2f36aa09 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260806 02:45:28.248951 21519 raft_consensus.cc:2454] T 10189388393d47ce900b44cbad3ff425 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:28.249558 21803 leader_election.cc:302] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:28.250002 22489 leader_election.cc:288] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:28.250190 22489 raft_consensus.cc:2781] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:28.250312 22489 raft_consensus.cc:686] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [term 1 LEADER]: Becoming Leader. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Running, Role: LEADER
I20260806 02:45:28.250600 22489 consensus_queue.cc:237] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } }
I20260806 02:45:28.251132 22599 ts_tablet_manager.cc:1918] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:45:28.251255 22599 log.cc:1196] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/2d8219e0438e46f783037537e687f945
I20260806 02:45:28.251562 21710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10189388393d47ce900b44cbad3ff425" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
I20260806 02:45:28.251776 22599 ts_tablet_manager.cc:1939] T 2d8219e0438e46f783037537e687f945 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:28.251840 21710 raft_consensus.cc:3037] T 10189388393d47ce900b44cbad3ff425 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:28.256273 21710 raft_consensus.cc:2454] T 10189388393d47ce900b44cbad3ff425 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:28.257375 22600 ts_tablet_manager.cc:1918] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:28.257534 22600 log.cc:1196] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/81dd98ae43b7429992d48eeaa9f17a70
I20260806 02:45:28.258045 22600 ts_tablet_manager.cc:1939] T 81dd98ae43b7429992d48eeaa9f17a70 P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:28.258092 22596 ts_tablet_manager.cc:1918] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:28.258224 22596 log.cc:1196] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/81dd98ae43b7429992d48eeaa9f17a70
W20260806 02:45:28.258776 21803 consensus_peers.cc:487] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 -> Peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Couldn't send request to peer 48b0c1e2106b4d9b941929ddc6d97dd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2d8219e0438e46f783037537e687f945. This is attempt 1: this message will repeat every 5th retry.
I20260806 02:45:28.259047 22600 tablet_replica.cc:330] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:28.259274 22600 raft_consensus.cc:2229] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:45:28.259565 22596 ts_tablet_manager.cc:1939] T 81dd98ae43b7429992d48eeaa9f17a70 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:28.259604 22600 raft_consensus.cc:2258] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:28.260154 22600 ts_tablet_manager.cc:1905] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:28.260730 22596 tablet_replica.cc:330] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:28.260944 22596 raft_consensus.cc:2229] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:28.261188 22596 raft_consensus.cc:2258] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:28.261816 22596 ts_tablet_manager.cc:1905] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:28.266601 22600 ts_tablet_manager.cc:1918] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:45:28.266724 22600 log.cc:1196] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/2d8219e0438e46f783037537e687f945
I20260806 02:45:28.267131 22600 ts_tablet_manager.cc:1939] T 2d8219e0438e46f783037537e687f945 P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:28.268833 22596 ts_tablet_manager.cc:1918] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:45:28.268986 22596 log.cc:1196] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/2d8219e0438e46f783037537e687f945
I20260806 02:45:28.269395 22596 ts_tablet_manager.cc:1939] T 2d8219e0438e46f783037537e687f945 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:28.324527 22548 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:28.324823 22548 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.325769 22548 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:28.326318 21519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cc50b1bfe414464b1863c4e26f295fb" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:28.326346 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cc50b1bfe414464b1863c4e26f295fb" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:28.326591 21519 raft_consensus.cc:2454] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:28.326663 21888 raft_consensus.cc:2454] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:28.327083 21617 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f; no voters:
I20260806 02:45:28.327385 22548 raft_consensus.cc:2781] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:28.327538 22548 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:28.327682 22548 raft_consensus.cc:3037] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:28.330037 22548 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.330626 22548 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:28.331219 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cc50b1bfe414464b1863c4e26f295fb" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:28.331278 21519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cc50b1bfe414464b1863c4e26f295fb" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:28.331467 21888 raft_consensus.cc:3037] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:28.331501 21519 raft_consensus.cc:3037] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:28.334365 21888 raft_consensus.cc:2454] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:28.334381 21519 raft_consensus.cc:2454] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:28.334764 21617 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:28.335045 22548 raft_consensus.cc:2781] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:28.335264 22548 raft_consensus.cc:686] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Becoming Leader. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Running, Role: LEADER
I20260806 02:45:28.335901 22548 consensus_queue.cc:237] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.348950 22548 raft_consensus.cc:482] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:28.349148 22548 raft_consensus.cc:504] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.349848 22548 leader_election.cc:288] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:28.350386 21519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c46742a5a44d54aba65eec2f36aa09" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:28.350453 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c46742a5a44d54aba65eec2f36aa09" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:28.350657 21519 raft_consensus.cc:2454] T 50c46742a5a44d54aba65eec2f36aa09 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:28.350694 21888 raft_consensus.cc:2454] T 50c46742a5a44d54aba65eec2f36aa09 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:28.351106 21617 leader_election.cc:302] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f; no voters:
I20260806 02:45:28.351411 22548 raft_consensus.cc:2781] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:28.351619 22548 raft_consensus.cc:482] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:28.351747 22548 raft_consensus.cc:3037] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:28.353955 22548 raft_consensus.cc:504] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.354444 22548 leader_election.cc:288] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:28.354939 21519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c46742a5a44d54aba65eec2f36aa09" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:28.355260 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c46742a5a44d54aba65eec2f36aa09" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:28.355448 21888 raft_consensus.cc:3037] T 50c46742a5a44d54aba65eec2f36aa09 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:28.358011 21888 raft_consensus.cc:2454] T 50c46742a5a44d54aba65eec2f36aa09 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:28.358366 21519 raft_consensus.cc:3037] T 50c46742a5a44d54aba65eec2f36aa09 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:28.358485 21617 leader_election.cc:302] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:28.358806 22548 raft_consensus.cc:2781] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:28.359424 22548 raft_consensus.cc:686] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Becoming Leader. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Running, Role: LEADER
I20260806 02:45:28.359813 22548 consensus_queue.cc:237] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.361277 21519 raft_consensus.cc:2454] T 50c46742a5a44d54aba65eec2f36aa09 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
W20260806 02:45:28.440789 21434 connection.cc:535] server connection from 127.0.0.1:33104 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:45:28.440919 21977 connection.cc:535] client connection to 127.20.60.62:33227 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:45:28.441666 21807 connection.cc:535] server connection from 127.0.0.1:56372 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:45:28.442278 21617 connection.cc:535] server connection from 127.0.0.1:38404 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 02:45:28.486332 20720 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
/tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.60:42309
--webserver_interface=127.20.60.60
--webserver_port=46389
--builtin_ntp_servers=127.20.60.20:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260806 02:45:28.633772 22648 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a225e002d607451a94b3686f10aeea9a)
I20260806 02:45:28.633984 22648 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:28.634706 22648 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:28.636826 22242 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:45:28.637135 22243 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:28.637244 22243 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a
I20260806 02:45:28.637530 22648 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260806 02:45:28.687088 22548 raft_consensus.cc:482] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:28.687333 22548 raft_consensus.cc:504] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.687953 22548 leader_election.cc:288] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:28.688798 21519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "293b93e6322842aaaa786570f2810548" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:28.688874 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "293b93e6322842aaaa786570f2810548" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5" is_pre_election: true
I20260806 02:45:28.689108 21519 raft_consensus.cc:2454] T 293b93e6322842aaaa786570f2810548 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:28.689118 21888 raft_consensus.cc:2454] T 293b93e6322842aaaa786570f2810548 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 0.
I20260806 02:45:28.689620 21617 leader_election.cc:302] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f; no voters:
I20260806 02:45:28.690075 22548 raft_consensus.cc:2781] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:28.690171 22548 raft_consensus.cc:482] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:28.690261 22548 raft_consensus.cc:3037] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:28.692975 22548 raft_consensus.cc:504] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.694043 21519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "293b93e6322842aaaa786570f2810548" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:28.694118 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "293b93e6322842aaaa786570f2810548" candidate_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebd96b903b944f3b06851324b737bd5"
I20260806 02:45:28.694288 21519 raft_consensus.cc:3037] T 293b93e6322842aaaa786570f2810548 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:28.694341 21888 raft_consensus.cc:3037] T 293b93e6322842aaaa786570f2810548 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:28.697188 21888 raft_consensus.cc:2454] T 293b93e6322842aaaa786570f2810548 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:28.697602 21519 raft_consensus.cc:2454] T 293b93e6322842aaaa786570f2810548 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88dc3ab284a64961b1fbdb5c53d1267f in term 1.
I20260806 02:45:28.697642 21617 leader_election.cc:302] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:28.697922 22497 raft_consensus.cc:2781] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:28.698504 22497 raft_consensus.cc:686] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Becoming Leader. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Running, Role: LEADER
I20260806 02:45:28.698778 22497 consensus_queue.cc:237] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:28.699756 22548 leader_election.cc:288] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [CANDIDATE]: Term 1 election: Requested vote from peers 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:45:28.879415 22644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:45:28.879797 22644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:45:28.879951 22644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:45:28.887846 22644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:45:28.888002 22644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:45:28.888105 22644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:45:28.888213 22644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:45:28.898983 22644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.60.20:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.60:42309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.60.60
--webserver_port=46389
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 01:55:54 UTC on ccycloud.quasar-csdhqu.root.comops.site
build id 4569536
ASAN enabled
I20260806 02:45:28.900679 22644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:45:28.903177 22644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:45:28.915953 22661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:28.916180 22663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:28.916456 22660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:45:28.916935 22644 server_base.cc:1034] running on GCE node
I20260806 02:45:28.917837 22644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:45:28.919482 22644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:45:28.920755 22644 hybrid_clock.cc:648] HybridClock initialized: now 1785984328920663 us; error 59 us; skew 500 ppm
I20260806 02:45:28.921205 22644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 02:45:28.928825 22644 webserver.cc:460] Webserver started at http://127.20.60.60:46389/ using document root <none> and password file <none>
I20260806 02:45:28.929629 22644 fs_manager.cc:359] Metadata directory not provided
I20260806 02:45:28.929795 22644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:45:28.935488 22644 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 02:45:28.938807 22670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:28.940186 22644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:45:28.940378 22644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data,/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/wal
uuid: "894be089543b4aa29b3edfcc5f3085e9"
format_stamp: "Formatted at 2026-08-06 02:45:18 on dist-test-slave-bw88"
I20260806 02:45:28.941404 22644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:45:28.958950 22644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:45:28.959980 22644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:45:28.960289 22644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:45:28.987229 22644 rpc_server.cc:307] RPC server started. Bound to: 127.20.60.60:42309
I20260806 02:45:28.987298 22732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.60.60:42309 every 8 connection(s)
I20260806 02:45:28.988795 22644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data/info.pb
I20260806 02:45:28.994221 20720 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu as pid 22644
I20260806 02:45:28.996323 22733 sys_catalog.cc:263] Verifying existing consensus state
I20260806 02:45:29.000531 22733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Bootstrap starting.
I20260806 02:45:29.022330 22733 log.cc:824] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Log is configured to *not* fsync() on all Append() calls
I20260806 02:45:29.031466 21761 heartbeater.cc:347] Connected to a master server at 127.20.60.60:42309
I20260806 02:45:29.051594 22560 consensus_queue.cc:1035] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:29.059049 22489 consensus_queue.cc:1035] T 10189388393d47ce900b44cbad3ff425 P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:29.061182 21761 heartbeater.cc:347] Connected to a master server at 127.20.60.60:42309
I20260806 02:45:29.072474 21950 heartbeater.cc:347] Connected to a master server at 127.20.60.60:42309
I20260806 02:45:29.074317 21581 heartbeater.cc:347] Connected to a master server at 127.20.60.60:42309
I20260806 02:45:29.118564 22733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=97 overwritten=0 applied=97 ignored=0} inserts{seen=64 ignored=0} mutations{seen=84 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 02:45:29.119223 22733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Bootstrap complete.
I20260806 02:45:29.126989 22733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:29.127919 22733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 894be089543b4aa29b3edfcc5f3085e9, State: Initialized, Role: FOLLOWER
I20260806 02:45:29.128620 22733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 97, Last appended: 4.97, Last appended by leader: 97, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:29.129523 22747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:29.129841 22747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:29.130844 22733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:45:29.148121 22766 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:29.148228 22766 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: loading cluster ID for follower catalog manager: success
I20260806 02:45:29.144402 22644 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:45:29.156143 22644 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:45:29.156174 22766 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: acquiring CA information for follower catalog manager: success
I20260806 02:45:29.159791 22771 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a225e002d607451a94b3686f10aeea9a)
I20260806 02:45:29.160005 22766 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 02:45:29.159996 22771 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:29.160727 22771 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:29.164327 22242 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:45:29.169070 22707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" candidate_term: 6 candidate_status { last_received { term: 5 index: 103 } } ignore_live_leader: false dest_uuid: "894be089543b4aa29b3edfcc5f3085e9" is_pre_election: true
I20260806 02:45:29.169814 22707 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfdbf4ef60014ea98bbcf9b6a9033f22 in term 4.
I20260806 02:45:29.170455 22243 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9, bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: a225e002d607451a94b3686f10aeea9a
I20260806 02:45:29.170783 22771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260806 02:45:29.170909 22771 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a225e002d607451a94b3686f10aeea9a)
I20260806 02:45:29.171010 22771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 5 FOLLOWER]: Advancing to term 6
I20260806 02:45:29.173051 22771 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:29.173763 22771 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 6 election: Requested vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
I20260806 02:45:29.174315 22707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" candidate_term: 6 candidate_status { last_received { term: 5 index: 103 } } ignore_live_leader: false dest_uuid: "894be089543b4aa29b3edfcc5f3085e9"
I20260806 02:45:29.174604 22707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 4 FOLLOWER]: Advancing to term 6
W20260806 02:45:29.178543 22242 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:45:29.181555 22707 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfdbf4ef60014ea98bbcf9b6a9033f22 in term 6.
I20260806 02:45:29.182005 22243 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9, bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: a225e002d607451a94b3686f10aeea9a
I20260806 02:45:29.182337 22771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 6 FOLLOWER]: Leader election won for term 6
I20260806 02:45:29.182869 22771 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 6 LEADER]: Becoming Leader. State: Replica: bfdbf4ef60014ea98bbcf9b6a9033f22, State: Running, Role: LEADER
I20260806 02:45:29.183456 22771 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 103, Committed index: 103, Last appended: 5.103, Last appended by leader: 103, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:29.185559 22776 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfdbf4ef60014ea98bbcf9b6a9033f22. Latest consensus state: current_term: 6 leader_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:29.185997 22776 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:29.186833 22780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:45:29.189756 22780 catalog_manager.cc:669] Loaded metadata for table default.table_e41f56f2e4a1439ca6b8f2bca460a670 [id=1bbd1184e7754c5bb9ffdf82791c73fb]
I20260806 02:45:29.190480 22780 catalog_manager.cc:669] Loaded metadata for table default.table_97ee2606f8aa47dcb2c67a9ae29dd3fb [id=370d5895c35f4c96878270e71cac2443]
I20260806 02:45:29.190817 22780 catalog_manager.cc:669] Loaded metadata for table default.table_ea1a38afd92a4b15bfabb80e58f8c644 [id=383ba93bcba140c1b1bd2b36f8720b86]
I20260806 02:45:29.191157 22780 catalog_manager.cc:669] Loaded metadata for table default.table_d7249617a95049389b90db3bd551ace4 [id=4179f72ea45043e39ddce8e93c3429e9]
I20260806 02:45:29.191488 22780 catalog_manager.cc:669] Loaded metadata for table default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e]
I20260806 02:45:29.191825 22780 catalog_manager.cc:669] Loaded metadata for table default.table_0febb4673dff4e20ae8124efba9627c8 [id=5cb271f99441401cad4fd2e40e943a6e]
I20260806 02:45:29.192147 22780 catalog_manager.cc:669] Loaded metadata for table default.table_21221d53bf9f4c46a5ffe671a1ffe424 [id=610932bdd3f94b4e886d586c28ff6249]
I20260806 02:45:29.192417 22780 catalog_manager.cc:669] Loaded metadata for table default.table_e8df6dde49954df7b57567fc0cd12872 [id=a0cffb03ae7641f8848b8d2aab0fca8b]
I20260806 02:45:29.192662 22780 catalog_manager.cc:669] Loaded metadata for table default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26]
I20260806 02:45:29.192929 22780 catalog_manager.cc:669] Loaded metadata for table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]
I20260806 02:45:29.193313 22780 catalog_manager.cc:669] Loaded metadata for table default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c]
I20260806 02:45:29.193780 22780 catalog_manager.cc:669] Loaded metadata for table default.table_00c06d2b7d6d40658a4390f4ee3bbab5 [id=c6a05485b0a54967ba8904aa969edd3f]
I20260806 02:45:29.194381 22780 catalog_manager.cc:669] Loaded metadata for table default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704]
I20260806 02:45:29.194844 22780 catalog_manager.cc:669] Loaded metadata for table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e]
I20260806 02:45:29.195242 22780 catalog_manager.cc:669] Loaded metadata for table default.table_c5be13d5ad2c4a479d2cc9074d14038f [id=f6939f0d45ae4f358d206a8cf2fe5db0]
I20260806 02:45:29.198562 22780 tablet_loader.cc:96] loaded metadata for tablet 043839a5eca143d5882f40faaab39692 (table default.table_21221d53bf9f4c46a5ffe671a1ffe424 [id=610932bdd3f94b4e886d586c28ff6249])
I20260806 02:45:29.199163 22780 tablet_loader.cc:96] loaded metadata for tablet 10189388393d47ce900b44cbad3ff425 (table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e])
I20260806 02:45:29.199723 22780 tablet_loader.cc:96] loaded metadata for tablet 237439c720344cfd97bd04bf8368a375 (table default.table_ea1a38afd92a4b15bfabb80e58f8c644 [id=383ba93bcba140c1b1bd2b36f8720b86])
I20260806 02:45:29.200058 22780 tablet_loader.cc:96] loaded metadata for tablet 293b93e6322842aaaa786570f2810548 (table default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26])
I20260806 02:45:29.200603 22780 tablet_loader.cc:96] loaded metadata for tablet 3196f6ee70d34d298b7db56d9759fab7 (table default.table_00c06d2b7d6d40658a4390f4ee3bbab5 [id=c6a05485b0a54967ba8904aa969edd3f])
I20260806 02:45:29.201143 22780 tablet_loader.cc:96] loaded metadata for tablet 3d44c58ba1f044b38f10da10cec410e3 (table default.table_e41f56f2e4a1439ca6b8f2bca460a670 [id=1bbd1184e7754c5bb9ffdf82791c73fb])
I20260806 02:45:29.201766 22780 tablet_loader.cc:96] loaded metadata for tablet 50c46742a5a44d54aba65eec2f36aa09 (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022])
I20260806 02:45:29.202088 22780 tablet_loader.cc:96] loaded metadata for tablet 6061f59aedec4e0a9b96239ebe6fa0df (table default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e])
I20260806 02:45:29.202445 22780 tablet_loader.cc:96] loaded metadata for tablet 62b1e9f4216845739fc757c975eb8c02 (table default.table_97ee2606f8aa47dcb2c67a9ae29dd3fb [id=370d5895c35f4c96878270e71cac2443])
I20260806 02:45:29.202751 22780 tablet_loader.cc:96] loaded metadata for tablet 6cc50b1bfe414464b1863c4e26f295fb (table default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c])
I20260806 02:45:29.203800 22780 tablet_loader.cc:96] loaded metadata for tablet a6e1420c19fc44b3901f653f137ec6c1 (table default.table_e8df6dde49954df7b57567fc0cd12872 [id=a0cffb03ae7641f8848b8d2aab0fca8b])
I20260806 02:45:29.204542 22780 tablet_loader.cc:96] loaded metadata for tablet ba32fbee948d4adf96164d36673a1db6 (table default.table_0febb4673dff4e20ae8124efba9627c8 [id=5cb271f99441401cad4fd2e40e943a6e])
I20260806 02:45:29.204931 22780 tablet_loader.cc:96] loaded metadata for tablet bea6ade491c6486393aca875767dc1b3 (table default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704])
I20260806 02:45:29.206203 22780 tablet_loader.cc:96] loaded metadata for tablet f9fd42a920b24117a4d687fec13ad17b (table default.table_d7249617a95049389b90db3bd551ace4 [id=4179f72ea45043e39ddce8e93c3429e9])
I20260806 02:45:29.206540 22780 tablet_loader.cc:96] loaded metadata for tablet fdc39b7dea2c43cebc730cffacd38d10 (table default.table_c5be13d5ad2c4a479d2cc9074d14038f [id=f6939f0d45ae4f358d206a8cf2fe5db0])
I20260806 02:45:29.206950 22780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:45:29.208519 22780 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:29.208626 22780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:45:29.210587 22780 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:45:29.211819 22780 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Loaded TSK: 0
I20260806 02:45:29.212742 22780 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:45:29.214597 22780 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260806 02:45:29.215925 22780 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: 29
I20260806 02:45:29.237268 22497 consensus_queue.cc:1035] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:29.243434 22695 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" instance_seqno: 1785984321670316) as {username='slave'} at 127.20.60.3:34523; Asking this server to re-register.
I20260806 02:45:29.245527 21950 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:29.247445 22696 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" instance_seqno: 1785984320818216) as {username='slave'} at 127.20.60.2:54649; Asking this server to re-register.
I20260806 02:45:29.247869 22652 consensus_queue.cc:1035] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:29.249775 22696 ts_manager.cc:194] Registered new tserver with Master: 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:29.247849 22275 catalog_manager.cc:5495] T 50c46742a5a44d54aba65eec2f36aa09 P 8ebd96b903b944f3b06851324b737bd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2). New cstate: current_term: 1 leader_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } }
I20260806 02:45:29.250087 22275 catalog_manager.cc:5587] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883) does not have the latest schema for tablet 50c46742a5a44d54aba65eec2f36aa09 (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]). Expected version 1 got 0
I20260806 02:45:29.251206 21761 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:29.253998 22696 ts_manager.cc:194] Registered new tserver with Master: 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:29.255051 22272 master_service.cc:946] ReplaceTablet: received request to replace tablet ba32fbee948d4adf96164d36673a1db6 from {username='slave'} at 127.0.0.1:41388
I20260806 02:45:29.258548 22269 master_service.cc:946] ReplaceTablet: received request to replace tablet ba32fbee948d4adf96164d36673a1db6 from {username='slave'} at 127.0.0.1:41388
I20260806 02:45:29.258889 22271 master_service.cc:946] ReplaceTablet: received request to replace tablet 6cc50b1bfe414464b1863c4e26f295fb from {username='slave'} at 127.0.0.1:41388
I20260806 02:45:29.259255 22277 master_service.cc:946] ReplaceTablet: received request to replace tablet 6061f59aedec4e0a9b96239ebe6fa0df from {username='slave'} at 127.0.0.1:41388
I20260806 02:45:29.259617 22262 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41388:
name: "default.Table_e412d99677f440648e5e0fea418116a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:29.262677 22260 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41388:
table { table_name: "default.Table_21221d53bf9f4c46a5ffe671a1ffe424" } new_table_name: "default.Table_fb2ecedbb20f49d2a659b25c01f24887" modify_external_catalogs: true
I20260806 02:45:29.263371 22263 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41388:
table { table_name: "default.Table_e24bd493305d46a4977ed544ce47cc83" } new_table_name: "default.Table_8e31b0866b344f85b9631a2e31b68af1" modify_external_catalogs: true
I20260806 02:45:29.263841 22267 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41388:
table { table_name: "default.Table_ddb47e155c91465f8e38147105f2a6bd" } new_table_name: "default.Table_a7570be8f5c74682ac8df015be8417b6" modify_external_catalogs: true
I20260806 02:45:29.275647 22696 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" instance_seqno: 1785984319928949) as {username='slave'} at 127.20.60.1:59571; Asking this server to re-register.
I20260806 02:45:29.277042 21581 heartbeater.cc:464] Registering TS with master...
W20260806 02:45:29.277873 22242 consensus_peers.cc:487] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 -> Peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Couldn't send request to peer a225e002d607451a94b3686f10aeea9a. Status: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 02:45:29.279256 22696 ts_manager.cc:194] Registered new tserver with Master: 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
F20260806 02:45:29.279386 22368 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 31): 31 DROP_TABLE default.table_1ff877d50df44dccb8be0241b847e8c9
*** Check failure stack trace: ***
I20260806 02:45:29.280690 22707 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Refusing update from remote peer bfdbf4ef60014ea98bbcf9b6a9033f22: Log matching property violated. Preceding OpId in replica: term: 4 index: 97. Preceding OpId from leader: term: 6 index: 105. (index mismatch)
I20260806 02:45:29.281881 22771 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 104, Last known committed idx: 97, Time since last communication: 0.000s
@ 0x7f3f122cc9ad google::LogMessage::Fail() at ??:0
@ 0x7f3f122cf88a google::LogMessage::SendToLog() at ??:0
@ 0x7f3f122cc4de google::LogMessage::Flush() at ??:0
@ 0x7f3f122cc6d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f3f27e6aba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f3f27e69901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f3f27e7066c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260806 02:45:29.308734 22747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfdbf4ef60014ea98bbcf9b6a9033f22. Latest consensus state: current_term: 6 leader_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:29.309093 22747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:29.310629 22790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:29.311067 22790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:29.311276 22790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:29.311496 22790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f3f27e704e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260806 02:45:29.320921 22787 mvcc.cc:205] Tried to move back new op lower bound from 7315391812720472064 to 7315391807512854528. Current Snapshot: MvccSnapshot[applied={T|T < 7315391812708605952}]
@ 0x7f3f14ef6eaf kudu::Thread::SuperviseThread() at ??:0
I20260806 02:45:29.323006 22277 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Sending DeleteTablet for 3 replicas of tablet 6061f59aedec4e0a9b96239ebe6fa0df
I20260806 02:45:29.324183 22277 catalog_manager.cc:6565] ReplaceTablet: tablet 6061f59aedec4e0a9b96239ebe6fa0df deleted and replaced by tablet 8243a6730a6e4939998608a4e4b63a18
I20260806 02:45:29.325119 22274 catalog_manager.cc:5587] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655) does not have the latest schema for tablet 50c46742a5a44d54aba65eec2f36aa09 (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]). Expected version 1 got 0
@ 0x7f3f181d06db start_thread at ??:0
@ 0x7f3f1116661f clone at ??:0
I20260806 02:45:29.427209 22652 consensus_queue.cc:1035] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:29.435669 21763 heartbeater.cc:503] Master 127.20.60.61:36797 was elected leader, sending a full tablet report...
I20260806 02:45:29.437318 22652 consensus_queue.cc:1035] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:29.463459 22548 consensus_queue.cc:1035] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:29.471347 22548 consensus_queue.cc:1035] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 02:45:29.575210 22802 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:45:29.325911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56380 (local address 127.20.60.3:43883)
0806 02:45:29.340686 (+ 14775us) server_negotiation.cc:206] Beginning negotiation
0806 02:45:29.340696 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 02:45:29.340726 (+ 30us) server_negotiation.cc:407] Connection header received
0806 02:45:29.340840 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:45:29.340848 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:45:29.340957 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:45:29.341084 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:45:29.574227 (+233143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56380: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":14586,"thread_start_us":15438,"threads_started":1}
W20260806 02:45:29.575212 22801 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:45:29.325755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38408 (local address 127.20.60.2:40655)
0806 02:45:29.343711 (+ 17956us) server_negotiation.cc:206] Beginning negotiation
0806 02:45:29.343715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:45:29.574165 (+230450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38408: BlockingRecv error: recv got EOF from 127.0.0.1:38408 (error 108)
Metrics: {"server-negotiator.queue_time_us":17849,"thread_start_us":17756,"threads_started":1}
W20260806 02:45:29.575219 22800 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:45:29.325624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33130 (local address 127.20.60.1:36593)
0806 02:45:29.340994 (+ 15370us) server_negotiation.cc:206] Beginning negotiation
0806 02:45:29.341002 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 02:45:29.341032 (+ 30us) server_negotiation.cc:407] Connection header received
0806 02:45:29.341108 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:45:29.341114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:45:29.341224 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:45:29.341323 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:45:29.574223 (+232900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33130: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":15236,"thread_start_us":15875,"threads_started":1}
W20260806 02:45:29.576416 21976 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.20.60.61:36797)
W20260806 02:45:29.577248 21976 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.20.60.61:36797)
W20260806 02:45:29.577804 21976 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.20.60.61:36797)
W20260806 02:45:29.578133 21976 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.20.60.61:36797)
W20260806 02:45:29.580870 21617 connection.cc:535] client connection to 127.20.60.61:36797 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 02:45:29.810379 22747 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:45:29.810559 22747 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:29.812367 22747 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:45:29.813352 22674 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:45:29.814685 22674 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:45:29.814915 22671 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:45:29.815065 22671 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:29.815346 22747 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260806 02:45:30.521692 22832 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:45:30.521904 22832 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:30.522693 22832 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:45:30.524859 22671 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:45:30.525041 22674 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:45:30.525238 22674 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:30.525521 22832 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260806 02:45:31.252952 22853 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:45:31.253233 22853 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:31.254089 22853 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:45:31.256335 22674 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:45:31.256531 22671 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:45:31.256701 22671 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:31.257009 22853 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260806 02:45:32.018908 20720 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
/tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.62:33227
--webserver_interface=127.20.60.62
--webserver_port=34225
--builtin_ntp_servers=127.20.60.20:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260806 02:45:32.250350 22875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:45:32.250653 22875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:32.251492 22875 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:45:32.254179 22671 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:45:32.254357 22674 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:45:32.254492 22671 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:32.254801 22875 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260806 02:45:32.384718 22868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:45:32.385076 22868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:45:32.385211 22868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:45:32.392370 22868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:45:32.392519 22868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:45:32.392611 22868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:45:32.392714 22868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:45:32.402303 22868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.60.20:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.62:33227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.60.62
--webserver_port=34225
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 01:55:54 UTC on ccycloud.quasar-csdhqu.root.comops.site
build id 4569536
ASAN enabled
I20260806 02:45:32.403745 22868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:45:32.405707 22868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:45:32.417918 22888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:32.418113 22887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:32.418128 22890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:45:32.419210 22868 server_base.cc:1034] running on GCE node
I20260806 02:45:32.420009 22868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:45:32.421489 22868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:45:32.422719 22868 hybrid_clock.cc:648] HybridClock initialized: now 1785984332422645 us; error 50 us; skew 500 ppm
I20260806 02:45:32.423138 22868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 02:45:32.425650 22868 webserver.cc:460] Webserver started at http://127.20.60.62:34225/ using document root <none> and password file <none>
I20260806 02:45:32.426371 22868 fs_manager.cc:359] Metadata directory not provided
I20260806 02:45:32.426541 22868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:45:32.431903 22868 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 02:45:32.435185 22897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:32.436662 22868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 02:45:32.436847 22868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data,/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
uuid: "a225e002d607451a94b3686f10aeea9a"
format_stamp: "Formatted at 2026-08-06 02:45:17 on dist-test-slave-bw88"
I20260806 02:45:32.437884 22868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:45:32.456645 22868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:45:32.457608 22868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:45:32.457906 22868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:45:32.477432 22868 rpc_server.cc:307] RPC server started. Bound to: 127.20.60.62:33227
I20260806 02:45:32.477521 22960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.60.62:33227 every 8 connection(s)
I20260806 02:45:32.479045 22868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data/info.pb
I20260806 02:45:32.484408 20720 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu as pid 22868
I20260806 02:45:32.487336 22961 sys_catalog.cc:263] Verifying existing consensus state
I20260806 02:45:32.491933 22961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Bootstrap starting.
I20260806 02:45:32.511736 21582 heartbeater.cc:347] Connected to a master server at 127.20.60.62:33227
I20260806 02:45:32.520854 22961 log.cc:824] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Log is configured to *not* fsync() on all Append() calls
I20260806 02:45:32.567126 21951 heartbeater.cc:347] Connected to a master server at 127.20.60.62:33227
I20260806 02:45:32.613749 22961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Bootstrap replayed 1/1 log segments. Stats: ops{read=103 overwritten=0 applied=103 ignored=0} inserts{seen=67 ignored=0} mutations{seen=90 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 02:45:32.614305 22961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Bootstrap complete.
I20260806 02:45:32.622877 22961 raft_consensus.cc:348] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:32.623806 22961 raft_consensus.cc:729] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: a225e002d607451a94b3686f10aeea9a, State: Initialized, Role: FOLLOWER
I20260806 02:45:32.624425 22961 consensus_queue.cc:260] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 103, Last appended: 5.103, Last appended by leader: 103, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:32.625250 22971 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:32.625684 22971 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:32.626497 22961 sys_catalog.cc:564] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:45:32.641239 22990 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:32.641373 22990 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: loading cluster ID for follower catalog manager: success
I20260806 02:45:32.650123 22990 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: acquiring CA information for follower catalog manager: success
I20260806 02:45:32.654441 22990 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 02:45:32.662106 22868 master_runner.cc:186] Error getting master registration for 127.20.60.61:36797: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111),
I20260806 02:45:32.668738 22868 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:45:32.698766 21762 heartbeater.cc:347] Connected to a master server at 127.20.60.62:33227
I20260806 02:45:32.703419 22918 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" instance_seqno: 1785984320818216) as {username='slave'} at 127.20.60.2:52619; Asking this server to re-register.
I20260806 02:45:32.704319 21762 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:32.706525 22918 ts_manager.cc:194] Registered new tserver with Master: 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:33.018018 22918 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" instance_seqno: 1785984319928949) as {username='slave'} at 127.20.60.1:48067; Asking this server to re-register.
I20260806 02:45:33.018891 21582 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:33.020830 22918 ts_manager.cc:194] Registered new tserver with Master: 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:33.072296 22918 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" instance_seqno: 1785984321670316) as {username='slave'} at 127.20.60.3:41883; Asking this server to re-register.
I20260806 02:45:33.073078 21951 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:33.074584 22918 ts_manager.cc:194] Registered new tserver with Master: 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:33.097574 22994 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:45:33.097779 22994 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:33.098489 22994 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:45:33.100854 22671 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:45:33.105719 22931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "894be089543b4aa29b3edfcc5f3085e9" candidate_term: 7 candidate_status { last_received { term: 6 index: 108 } } ignore_live_leader: false dest_uuid: "a225e002d607451a94b3686f10aeea9a" is_pre_election: true
I20260806 02:45:33.106282 22931 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 894be089543b4aa29b3edfcc5f3085e9 in term 5.
I20260806 02:45:33.106920 22674 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a; no voters: bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:33.107228 22994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260806 02:45:33.107355 22994 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:45:33.107455 22994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 6 FOLLOWER]: Advancing to term 7
I20260806 02:45:33.109653 22994 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:33.110263 22994 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 election: Requested vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
I20260806 02:45:33.110720 22931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "894be089543b4aa29b3edfcc5f3085e9" candidate_term: 7 candidate_status { last_received { term: 6 index: 108 } } ignore_live_leader: false dest_uuid: "a225e002d607451a94b3686f10aeea9a"
I20260806 02:45:33.110988 22931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 5 FOLLOWER]: Advancing to term 7
W20260806 02:45:33.111665 22671 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:45:33.115265 22931 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 894be089543b4aa29b3edfcc5f3085e9 in term 7.
I20260806 02:45:33.115681 22674 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a; no voters: bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:33.115958 22994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 7 FOLLOWER]: Leader election won for term 7
I20260806 02:45:33.116439 22994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 7 LEADER]: Becoming Leader. State: Replica: 894be089543b4aa29b3edfcc5f3085e9, State: Running, Role: LEADER
I20260806 02:45:33.116938 22994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 108, Committed index: 108, Last appended: 6.108, Last appended by leader: 108, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:33.118729 22998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 894be089543b4aa29b3edfcc5f3085e9. Latest consensus state: current_term: 7 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:33.119205 22998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:33.119869 23000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:45:33.122900 23000 catalog_manager.cc:669] Loaded metadata for table default.table_e41f56f2e4a1439ca6b8f2bca460a670 [id=1bbd1184e7754c5bb9ffdf82791c73fb]
I20260806 02:45:33.123611 23000 catalog_manager.cc:669] Loaded metadata for table default.table_97ee2606f8aa47dcb2c67a9ae29dd3fb [id=370d5895c35f4c96878270e71cac2443]
I20260806 02:45:33.123983 23000 catalog_manager.cc:669] Loaded metadata for table default.table_ea1a38afd92a4b15bfabb80e58f8c644 [id=383ba93bcba140c1b1bd2b36f8720b86]
I20260806 02:45:33.124315 23000 catalog_manager.cc:669] Loaded metadata for table default.table_d7249617a95049389b90db3bd551ace4 [id=4179f72ea45043e39ddce8e93c3429e9]
I20260806 02:45:33.124642 23000 catalog_manager.cc:669] Loaded metadata for table default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e]
I20260806 02:45:33.124976 23000 catalog_manager.cc:669] Loaded metadata for table default.table_0febb4673dff4e20ae8124efba9627c8 [id=5cb271f99441401cad4fd2e40e943a6e]
I20260806 02:45:33.125288 23000 catalog_manager.cc:669] Loaded metadata for table default.table_21221d53bf9f4c46a5ffe671a1ffe424 [id=610932bdd3f94b4e886d586c28ff6249]
I20260806 02:45:33.125660 23000 catalog_manager.cc:669] Loaded metadata for table default.table_e8df6dde49954df7b57567fc0cd12872 [id=a0cffb03ae7641f8848b8d2aab0fca8b]
I20260806 02:45:33.125978 23000 catalog_manager.cc:669] Loaded metadata for table default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26]
I20260806 02:45:33.126304 23000 catalog_manager.cc:669] Loaded metadata for table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]
I20260806 02:45:33.126616 23000 catalog_manager.cc:669] Loaded metadata for table default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c]
I20260806 02:45:33.126965 23000 catalog_manager.cc:669] Loaded metadata for table default.table_00c06d2b7d6d40658a4390f4ee3bbab5 [id=c6a05485b0a54967ba8904aa969edd3f]
I20260806 02:45:33.127389 23000 catalog_manager.cc:669] Loaded metadata for table default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704]
I20260806 02:45:33.127725 23000 catalog_manager.cc:669] Loaded metadata for table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e]
I20260806 02:45:33.128044 23000 catalog_manager.cc:669] Loaded metadata for table default.table_c5be13d5ad2c4a479d2cc9074d14038f [id=f6939f0d45ae4f358d206a8cf2fe5db0]
I20260806 02:45:33.130409 23000 tablet_loader.cc:96] loaded metadata for tablet 043839a5eca143d5882f40faaab39692 (table default.table_21221d53bf9f4c46a5ffe671a1ffe424 [id=610932bdd3f94b4e886d586c28ff6249])
I20260806 02:45:33.130896 23000 tablet_loader.cc:96] loaded metadata for tablet 10189388393d47ce900b44cbad3ff425 (table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e])
I20260806 02:45:33.131366 23000 tablet_loader.cc:96] loaded metadata for tablet 237439c720344cfd97bd04bf8368a375 (table default.table_ea1a38afd92a4b15bfabb80e58f8c644 [id=383ba93bcba140c1b1bd2b36f8720b86])
I20260806 02:45:33.131610 23000 tablet_loader.cc:96] loaded metadata for tablet 290cc76d8c174a1fa3645f3bfe754b52 (table default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c])
I20260806 02:45:33.131912 23000 tablet_loader.cc:96] loaded metadata for tablet 293b93e6322842aaaa786570f2810548 (table default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26])
I20260806 02:45:33.132360 23000 tablet_loader.cc:96] loaded metadata for tablet 3196f6ee70d34d298b7db56d9759fab7 (table default.table_00c06d2b7d6d40658a4390f4ee3bbab5 [id=c6a05485b0a54967ba8904aa969edd3f])
I20260806 02:45:33.132802 23000 tablet_loader.cc:96] loaded metadata for tablet 3d44c58ba1f044b38f10da10cec410e3 (table default.table_e41f56f2e4a1439ca6b8f2bca460a670 [id=1bbd1184e7754c5bb9ffdf82791c73fb])
I20260806 02:45:33.133216 23000 tablet_loader.cc:96] loaded metadata for tablet 50c46742a5a44d54aba65eec2f36aa09 (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022])
I20260806 02:45:33.133777 23000 tablet_loader.cc:96] loaded metadata for tablet 62b1e9f4216845739fc757c975eb8c02 (table default.table_97ee2606f8aa47dcb2c67a9ae29dd3fb [id=370d5895c35f4c96878270e71cac2443])
I20260806 02:45:33.134429 23000 tablet_loader.cc:96] loaded metadata for tablet 8243a6730a6e4939998608a4e4b63a18 (table default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e])
I20260806 02:45:33.134903 23000 tablet_loader.cc:96] loaded metadata for tablet a6e1420c19fc44b3901f653f137ec6c1 (table default.table_e8df6dde49954df7b57567fc0cd12872 [id=a0cffb03ae7641f8848b8d2aab0fca8b])
I20260806 02:45:33.135557 23000 tablet_loader.cc:96] loaded metadata for tablet bea6ade491c6486393aca875767dc1b3 (table default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704])
I20260806 02:45:33.136193 23000 tablet_loader.cc:96] loaded metadata for tablet cbfd41ea3624457da227733bea68ffc5 (table default.table_0febb4673dff4e20ae8124efba9627c8 [id=5cb271f99441401cad4fd2e40e943a6e])
I20260806 02:45:33.136952 23000 tablet_loader.cc:96] loaded metadata for tablet f9fd42a920b24117a4d687fec13ad17b (table default.table_d7249617a95049389b90db3bd551ace4 [id=4179f72ea45043e39ddce8e93c3429e9])
I20260806 02:45:33.137269 23000 tablet_loader.cc:96] loaded metadata for tablet fdc39b7dea2c43cebc730cffacd38d10 (table default.table_c5be13d5ad2c4a479d2cc9074d14038f [id=f6939f0d45ae4f358d206a8cf2fe5db0])
I20260806 02:45:33.137724 23000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:45:33.139153 23000 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:33.139257 23000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:45:33.140923 23000 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:45:33.142474 23000 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Loaded TSK: 0
I20260806 02:45:33.143539 23000 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:45:33.144847 23000 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260806 02:45:33.146358 23000 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: 29
I20260806 02:45:33.169129 22931 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Refusing update from remote peer 894be089543b4aa29b3edfcc5f3085e9: Log matching property violated. Preceding OpId in replica: term: 5 index: 103. Preceding OpId from leader: term: 7 index: 110. (index mismatch)
W20260806 02:45:33.169486 22671 consensus_peers.cc:487] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 -> Peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Couldn't send request to peer bfdbf4ef60014ea98bbcf9b6a9033f22. Status: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 02:45:33.170032 22998 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 109, Last known committed idx: 103, Time since last communication: 0.000s
I20260806 02:45:33.179128 22695 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34432:
name: "default.Table_3945346dd2514eddba4dfc965eb5d995"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:33.177193 22690 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34432:
name: "default.Table_8ee8dd77f08a4c9d9e3d4f345a9a6bbf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:33.180414 22696 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34432:
name: "default.Table_468b3a35d3fb4572bc7b89d436d4e319"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:33.182166 22687 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34432:
table { table_name: "default.Table_0febb4673dff4e20ae8124efba9627c8" } modify_external_catalogs: true
I20260806 02:45:33.182524 22687 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34432:
table { table_name: "default.Table_0febb4673dff4e20ae8124efba9627c8" } modify_external_catalogs: true
I20260806 02:45:33.178773 22692 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34432:
name: "default.Table_cac77bd182314ee2a2068374ddeca8d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:33.179267 22685 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34432:
name: "default.Table_6d994a6895c3485cb404565731c2e7fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:33.184135 22684 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34432:
table { table_name: "default.Table_e24bd493305d46a4977ed544ce47cc83" } new_table_name: "default.Table_8e31b0866b344f85b9631a2e31b68af1" modify_external_catalogs: true
I20260806 02:45:33.181821 22691 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34432:
name: "default.Table_8a4c223baf6149efb9e759a304f00edd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:33.182940 22686 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34432:
name: "default.Table_5ed544635d9c480cb8c61f25b7f0f84f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:33.184904 23005 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:33.185302 23005 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:33.185690 23003 mvcc.cc:205] Tried to move back new op lower bound from 7315391812720472064 to 7315391812339978240. Current Snapshot: MvccSnapshot[applied={T|T < 7315391812720472064 or (T in {7315391812720472064})}]
I20260806 02:45:33.185607 23002 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 894be089543b4aa29b3edfcc5f3085e9. Latest consensus state: current_term: 7 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:33.185984 23002 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:33.177023 22693 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34432:
name: "default.Table_1ff877d50df44dccb8be0241b847e8c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260806 02:45:33.190758 22764 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 31): 31 DROP_TABLE default.table_1ff877d50df44dccb8be0241b847e8c9
*** Check failure stack trace: ***
I20260806 02:45:33.190973 22998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
@ 0x7f5906b849ad google::LogMessage::Fail() at ??:0
@ 0x7f5906b8788a google::LogMessage::SendToLog() at ??:0
@ 0x7f5906b844de google::LogMessage::Flush() at ??:0
I20260806 02:45:33.197570 22998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:33.198351 23002 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "894be089543b4aa29b3edfcc5f3085e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:33.198582 23002 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f5906b846d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f591c722ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f591c721901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f591c72866c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f591c7284e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260806 02:45:33.221041 21863 tablet_service.cc:1467] Processing CreateTablet for tablet cbfd41ea3624457da227733bea68ffc5 (DEFAULT_TABLE table=default.table_0febb4673dff4e20ae8124efba9627c8 [id=5cb271f99441401cad4fd2e40e943a6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:33.221583 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbfd41ea3624457da227733bea68ffc5. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260806 02:45:33.223515 21862 tablet_service.cc:1467] Processing CreateTablet for tablet 290cc76d8c174a1fa3645f3bfe754b52 (DEFAULT_TABLE table=default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:33.223902 21862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 290cc76d8c174a1fa3645f3bfe754b52. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f59097aeeaf kudu::Thread::SuperviseThread() at ??:0
I20260806 02:45:33.227281 21865 tablet_service.cc:1467] Processing CreateTablet for tablet 8243a6730a6e4939998608a4e4b63a18 (DEFAULT_TABLE table=default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:33.227644 21865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8243a6730a6e4939998608a4e4b63a18. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f590ca886db start_thread at ??:0
I20260806 02:45:33.228703 23019 tablet_bootstrap.cc:492] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
@ 0x7f5905a1e61f clone at ??:0
I20260806 02:45:33.231043 23019 tablet_bootstrap.cc:654] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:33.240562 23019 tablet_bootstrap.cc:492] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:33.240708 23019 ts_tablet_manager.cc:1397] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260806 02:45:33.241652 23019 raft_consensus.cc:348] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:33.241932 23019 raft_consensus.cc:374] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:33.242043 23019 raft_consensus.cc:729] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:33.242307 23019 consensus_queue.cc:260] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:33.242940 23019 ts_tablet_manager.cc:1428] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 02:45:33.243307 23019 tablet_bootstrap.cc:492] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:33.245843 23019 tablet_bootstrap.cc:654] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:33.247679 23019 tablet_bootstrap.cc:492] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:33.247819 23019 ts_tablet_manager.cc:1397] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 02:45:33.248488 23019 raft_consensus.cc:348] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:33.248692 23019 raft_consensus.cc:374] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:33.248760 23019 raft_consensus.cc:729] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:33.248994 23019 consensus_queue.cc:260] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:33.249622 23019 ts_tablet_manager.cc:1428] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 02:45:33.249977 23019 tablet_bootstrap.cc:492] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
I20260806 02:45:33.251642 23019 tablet_bootstrap.cc:654] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:33.257411 23019 tablet_bootstrap.cc:492] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:33.257562 23019 ts_tablet_manager.cc:1397] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 02:45:33.258347 23019 raft_consensus.cc:348] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:33.258598 23019 raft_consensus.cc:374] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:33.258692 23019 raft_consensus.cc:729] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:33.258942 23019 consensus_queue.cc:260] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:33.259739 23019 ts_tablet_manager.cc:1428] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260806 02:45:33.425045 23014 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:45:33.206988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37534 (local address 127.20.60.2:40655)
0806 02:45:33.217818 (+ 10830us) server_negotiation.cc:206] Beginning negotiation
0806 02:45:33.217827 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 02:45:33.217847 (+ 20us) server_negotiation.cc:407] Connection header received
0806 02:45:33.218710 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:45:33.218718 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:45:33.218812 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:45:33.219638 (+ 826us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:45:33.219859 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:45:33.221740 (+ 1881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:45:33.235558 (+ 13818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:45:33.237212 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:45:33.237396 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:45:33.237537 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 02:45:33.424644 (+187107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37534: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":10685,"thread_start_us":10531,"threads_started":1}
W20260806 02:45:33.425225 23017 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:45:33.208146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49412 (local address 127.20.60.1:36593)
0806 02:45:33.219410 (+ 11264us) server_negotiation.cc:206] Beginning negotiation
0806 02:45:33.219417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:45:33.219438 (+ 21us) server_negotiation.cc:407] Connection header received
0806 02:45:33.219528 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:45:33.219536 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:45:33.219649 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:45:33.220631 (+ 982us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:45:33.220833 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:45:33.221950 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:45:33.234779 (+ 12829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:45:33.238650 (+ 3871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:45:33.238788 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:45:33.238912 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 02:45:33.424702 (+185790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49412: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":11132,"thread_start_us":11173,"threads_started":1}
W20260806 02:45:33.426741 21978 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.20.60.60:42309)
W20260806 02:45:33.427189 21978 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.20.60.60:42309)
W20260806 02:45:33.427652 21978 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.20.60.60:42309)
W20260806 02:45:33.427892 21978 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.20.60.60:42309)
W20260806 02:45:33.428746 21806 connection.cc:535] client connection to 127.20.60.60:42309 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:45:33.431738 22970 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:45:33.430880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.60.60:42309 (local address 127.20.60.3:43181)
0806 02:45:33.431181 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 02:45:33.431200 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 02:45:33.431339 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:45:33.431501 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":161}
W20260806 02:45:33.433399 22822 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:45:33.431131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.60.60:42309 (local address 127.0.0.1:45788)
0806 02:45:33.431371 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 02:45:33.431481 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 02:45:33.556253 22828 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:33.556497 22828 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:33.557245 22828 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:45:33.558241 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:45:33.558457 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:45:33.558569 21803 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:33.558832 22828 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:45:33.698305 23002 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 894be089543b4aa29b3edfcc5f3085e9)
I20260806 02:45:33.698570 23002 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:33.700672 23002 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:33.702605 22898 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:45:33.703881 22900 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
W20260806 02:45:33.704164 22898 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:45:33.704358 22898 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a225e002d607451a94b3686f10aeea9a; no voters: 894be089543b4aa29b3edfcc5f3085e9, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:33.705090 23002 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260806 02:45:33.737592 22828 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:33.737804 22828 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:33.738544 22828 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:45:33.739517 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:45:33.739773 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:45:33.739876 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:33.740231 22828 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:45:33.848755 22828 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:33.848973 22828 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:33.849774 22828 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:45:33.850678 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:45:33.850873 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:45:33.851049 21806 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:33.851308 22828 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:45:34.378137 23042 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 894be089543b4aa29b3edfcc5f3085e9)
I20260806 02:45:34.378358 23042 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:34.379309 23042 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:34.381718 22900 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
W20260806 02:45:34.381882 22898 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:45:34.382040 22898 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a225e002d607451a94b3686f10aeea9a; no voters: 894be089543b4aa29b3edfcc5f3085e9, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:34.382412 23042 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260806 02:45:34.973116 23047 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 894be089543b4aa29b3edfcc5f3085e9)
I20260806 02:45:34.973428 23047 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:34.974310 23047 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:34.976529 22898 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:45:34.976797 22900 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:34.976941 22900 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a225e002d607451a94b3686f10aeea9a; no voters: 894be089543b4aa29b3edfcc5f3085e9, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:34.977246 23047 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260806 02:45:34.997215 20720 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu pid 22868
I20260806 02:45:35.011042 22955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 02:45:35.127146 20720 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu with pid 22868
I20260806 02:45:36.072135 20720 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
/tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.62:33227
--webserver_interface=127.20.60.62
--webserver_port=34225
--builtin_ntp_servers=127.20.60.20:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260806 02:45:36.441625 23059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:45:36.441924 23059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:45:36.442072 23059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:45:36.449672 23059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:45:36.449779 23059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:45:36.449859 23059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:45:36.449939 23059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:45:36.459893 23059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.60.20:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.62:33227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.60.62
--webserver_port=34225
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 01:55:54 UTC on ccycloud.quasar-csdhqu.root.comops.site
build id 4569536
ASAN enabled
I20260806 02:45:36.461670 23059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:45:36.463539 23059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:45:36.476101 23065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:36.476073 23066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:36.476505 23070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:45:36.476888 23059 server_base.cc:1034] running on GCE node
I20260806 02:45:36.477797 23059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:45:36.479290 23059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:45:36.480537 23059 hybrid_clock.cc:648] HybridClock initialized: now 1785984336480436 us; error 40 us; skew 500 ppm
I20260806 02:45:36.480897 23059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 02:45:36.483358 23059 webserver.cc:460] Webserver started at http://127.20.60.62:34225/ using document root <none> and password file <none>
I20260806 02:45:36.484127 23059 fs_manager.cc:359] Metadata directory not provided
I20260806 02:45:36.484256 23059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:45:36.489681 23059 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 02:45:36.495031 23077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:36.496536 23059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 02:45:36.496718 23059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data,/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
uuid: "a225e002d607451a94b3686f10aeea9a"
format_stamp: "Formatted at 2026-08-06 02:45:17 on dist-test-slave-bw88"
I20260806 02:45:36.497833 23059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:45:36.522214 23059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:45:36.523186 23059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:45:36.523510 23059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:45:36.543911 23059 rpc_server.cc:307] RPC server started. Bound to: 127.20.60.62:33227
I20260806 02:45:36.543931 23139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.60.62:33227 every 8 connection(s)
I20260806 02:45:36.545620 23059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data/info.pb
I20260806 02:45:36.552701 23140 sys_catalog.cc:263] Verifying existing consensus state
I20260806 02:45:36.555738 20720 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu as pid 23059
I20260806 02:45:36.556921 23140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Bootstrap starting.
I20260806 02:45:36.580627 23140 log.cc:824] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Log is configured to *not* fsync() on all Append() calls
I20260806 02:45:36.670416 21582 heartbeater.cc:347] Connected to a master server at 127.20.60.62:33227
I20260806 02:45:36.671500 23140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Bootstrap replayed 1/1 log segments. Stats: ops{read=110 overwritten=0 applied=110 ignored=0} inserts{seen=70 ignored=0} mutations{seen=97 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 02:45:36.672137 23140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Bootstrap complete.
I20260806 02:45:36.680084 23140 raft_consensus.cc:348] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:36.681021 23140 raft_consensus.cc:729] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: a225e002d607451a94b3686f10aeea9a, State: Initialized, Role: FOLLOWER
I20260806 02:45:36.681716 23140 consensus_queue.cc:260] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 110, Last appended: 7.110, Last appended by leader: 110, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:36.682569 23145 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:36.682984 23145 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:36.683696 23140 sys_catalog.cc:564] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:45:36.700197 23161 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:36.700306 23161 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: loading cluster ID for follower catalog manager: success
I20260806 02:45:36.708557 23161 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: acquiring CA information for follower catalog manager: success
I20260806 02:45:36.713766 23161 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 02:45:36.720533 23059 master_runner.cc:186] Error getting master registration for 127.20.60.61:36797: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111),
I20260806 02:45:36.729688 23059 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:45:36.730921 21762 heartbeater.cc:347] Connected to a master server at 127.20.60.62:33227
I20260806 02:45:36.734110 23100 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" instance_seqno: 1785984320818216) as {username='slave'} at 127.20.60.2:45845; Asking this server to re-register.
I20260806 02:45:36.734841 21762 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:36.737036 23100 ts_manager.cc:194] Registered new tserver with Master: 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:36.784080 21951 heartbeater.cc:347] Connected to a master server at 127.20.60.62:33227
I20260806 02:45:36.786132 23100 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" instance_seqno: 1785984321670316) as {username='slave'} at 127.20.60.3:43977; Asking this server to re-register.
I20260806 02:45:36.786911 21951 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:36.788678 23100 ts_manager.cc:194] Registered new tserver with Master: 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:37.062456 23145 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:37.062851 23145 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:37.065428 23145 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:37.066202 23079 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:45:37.068055 23079 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:45:37.068351 23081 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:37.068537 23081 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a225e002d607451a94b3686f10aeea9a; no voters: 894be089543b4aa29b3edfcc5f3085e9, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:37.068938 23145 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260806 02:45:37.174912 23100 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" instance_seqno: 1785984319928949) as {username='slave'} at 127.20.60.1:56845; Asking this server to re-register.
I20260806 02:45:37.175642 21582 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:37.177265 23100 ts_manager.cc:194] Registered new tserver with Master: 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:37.605746 23169 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:37.605990 23169 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:37.607594 23169 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:37.609479 23079 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:45:37.609740 23081 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:37.609863 23081 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a225e002d607451a94b3686f10aeea9a; no voters: 894be089543b4aa29b3edfcc5f3085e9, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:37.610152 23169 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260806 02:45:37.709545 22828 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:37.709726 22828 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:37.710455 22828 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:45:37.711411 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:45:37.711620 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:45:37.711817 21803 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:37.712102 22828 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:45:37.893556 22828 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:37.893749 22828 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:37.894419 22828 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:45:37.895483 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:45:37.895673 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:45:37.895843 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:37.896119 22828 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:45:38.315413 23175 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:38.315711 22828 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:38.315693 23175 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:38.315884 22828 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:38.316561 23175 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
I20260806 02:45:38.316777 22828 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:45:38.317948 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:45:38.318598 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:45:38.318816 21806 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:38.319137 22828 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:45:38.320027 23081 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
W20260806 02:45:38.320312 23079 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:45:38.320437 23079 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a225e002d607451a94b3686f10aeea9a; no voters: 894be089543b4aa29b3edfcc5f3085e9, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:38.320679 23175 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260806 02:45:39.613770 23182 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:39.613991 23182 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:39.614753 23182 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:39.616931 23081 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
W20260806 02:45:39.617113 23079 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:45:39.617264 23079 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a225e002d607451a94b3686f10aeea9a; no voters: 894be089543b4aa29b3edfcc5f3085e9, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:39.617533 23182 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260806 02:45:39.767181 20720 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
/tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.61:36797
--webserver_interface=127.20.60.61
--webserver_port=46303
--builtin_ntp_servers=127.20.60.20:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260806 02:45:40.127810 23188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:45:40.128204 23188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:45:40.128372 23188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:45:40.136457 23188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:45:40.136593 23188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:45:40.136685 23188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:45:40.136780 23188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:45:40.146488 23188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.60.20:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.61:36797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.60.61
--webserver_port=46303
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 01:55:54 UTC on ccycloud.quasar-csdhqu.root.comops.site
build id 4569536
ASAN enabled
I20260806 02:45:40.147992 23188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:45:40.150097 23188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:45:40.162933 23198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:40.162971 23200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:40.163293 23202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:45:40.163969 23188 server_base.cc:1034] running on GCE node
I20260806 02:45:40.164952 23188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:45:40.166603 23188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:45:40.167860 23188 hybrid_clock.cc:648] HybridClock initialized: now 1785984340167783 us; error 48 us; skew 500 ppm
I20260806 02:45:40.168217 23188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 02:45:40.174340 23188 webserver.cc:460] Webserver started at http://127.20.60.61:46303/ using document root <none> and password file <none>
I20260806 02:45:40.175112 23188 fs_manager.cc:359] Metadata directory not provided
I20260806 02:45:40.175261 23188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:45:40.181949 23188 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 02:45:40.185667 23209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:40.187240 23188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:45:40.187420 23188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data,/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22"
format_stamp: "Formatted at 2026-08-06 02:45:17 on dist-test-slave-bw88"
I20260806 02:45:40.188409 23188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:45:40.209154 23188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:45:40.210157 23188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:45:40.210481 23188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:45:40.230641 23188 rpc_server.cc:307] RPC server started. Bound to: 127.20.60.61:36797
I20260806 02:45:40.230654 23270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.60.61:36797 every 8 connection(s)
I20260806 02:45:40.232227 23188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data/info.pb
I20260806 02:45:40.234707 20720 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu as pid 23188
I20260806 02:45:40.244131 23271 sys_catalog.cc:263] Verifying existing consensus state
I20260806 02:45:40.250277 23271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Bootstrap starting.
I20260806 02:45:40.268440 23271 log.cc:824] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Log is configured to *not* fsync() on all Append() calls
I20260806 02:45:40.285648 21952 heartbeater.cc:347] Connected to a master server at 127.20.60.61:36797
I20260806 02:45:40.350183 23271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Bootstrap replayed 1/1 log segments. Stats: ops{read=109 overwritten=1 applied=106 ignored=0} inserts{seen=68 ignored=0} mutations{seen=92 ignored=0} orphaned_commits=0. Pending: 2 replicates
I20260806 02:45:40.350863 23271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Bootstrap complete.
I20260806 02:45:40.359737 23271 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 6 FOLLOWER]: Replica starting. Triggering 2 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:40.363258 23271 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfdbf4ef60014ea98bbcf9b6a9033f22, State: Initialized, Role: FOLLOWER
I20260806 02:45:40.363996 23271 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 106, Last appended: 6.108, Last appended by leader: 108, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:40.364885 23277 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:40.365259 23277 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:40.366065 23271 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:45:40.384444 23295 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:40.384605 23295 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: loading cluster ID for follower catalog manager: success
I20260806 02:45:40.390746 23295 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: acquiring CA information for follower catalog manager: success
I20260806 02:45:40.396490 23295 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 02:45:40.401858 23188 master_runner.cc:186] Error getting master registration for 127.20.60.60:42309: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111),
I20260806 02:45:40.410104 23188 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:45:40.532176 21583 heartbeater.cc:347] Connected to a master server at 127.20.60.61:36797
I20260806 02:45:40.535787 23231 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" instance_seqno: 1785984319928949) as {username='slave'} at 127.20.60.1:52297; Asking this server to re-register.
I20260806 02:45:40.536723 21583 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:40.538944 23231 ts_manager.cc:194] Registered new tserver with Master: 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:40.724413 21763 heartbeater.cc:347] Connected to a master server at 127.20.60.61:36797
I20260806 02:45:40.724682 21763 heartbeater.cc:503] Master 127.20.60.61:36797 was elected leader, sending a full tablet report...
I20260806 02:45:40.727638 23231 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" instance_seqno: 1785984320818216) as {username='slave'} at 127.20.60.2:56151; Asking this server to re-register.
I20260806 02:45:40.728482 21763 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:40.730261 23231 ts_manager.cc:194] Registered new tserver with Master: 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:40.791411 23231 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" instance_seqno: 1785984321670316) as {username='slave'} at 127.20.60.3:47483; Asking this server to re-register.
I20260806 02:45:40.792213 21952 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:40.793648 23231 ts_manager.cc:194] Registered new tserver with Master: 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:40.884990 23302 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:40.885365 23302 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:40.887410 23302 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:40.888222 23212 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:40.888201 23112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" candidate_term: 7 candidate_status { last_received { term: 6 index: 108 } } ignore_live_leader: false dest_uuid: "a225e002d607451a94b3686f10aeea9a" is_pre_election: true
I20260806 02:45:40.888818 23112 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bfdbf4ef60014ea98bbcf9b6a9033f22 in current term 7: Already voted for candidate 894be089543b4aa29b3edfcc5f3085e9 in this term.
W20260806 02:45:40.889791 23212 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:40.889917 23212 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a
I20260806 02:45:40.890302 23302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 6 FOLLOWER]: Advancing to term 7
I20260806 02:45:40.894656 23302 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260806 02:45:41.615622 23312 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:41.615806 23312 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:41.616664 23312 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
I20260806 02:45:41.617085 23112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" candidate_term: 8 candidate_status { last_received { term: 6 index: 108 } } ignore_live_leader: false dest_uuid: "a225e002d607451a94b3686f10aeea9a" is_pre_election: true
I20260806 02:45:41.617537 23112 raft_consensus.cc:2396] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bfdbf4ef60014ea98bbcf9b6a9033f22 for term 8 because replica has last-logged OpId of term: 7 index: 110, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 108.
W20260806 02:45:41.618916 23212 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:41.619092 23212 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a
I20260806 02:45:41.619388 23312 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260806 02:45:41.629405 22828 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:41.629583 22828 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:41.630201 22828 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:45:41.631155 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:45:41.631366 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:45:41.631479 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:41.631774 22828 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:45:41.687069 23316 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:41.687373 23316 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:41.688237 23316 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:41.694025 23081 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:41.698014 23244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a225e002d607451a94b3686f10aeea9a" candidate_term: 8 candidate_status { last_received { term: 7 index: 110 } } ignore_live_leader: false dest_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" is_pre_election: true
I20260806 02:45:41.698473 23244 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a225e002d607451a94b3686f10aeea9a in term 7.
I20260806 02:45:41.699021 23079 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: 894be089543b4aa29b3edfcc5f3085e9
I20260806 02:45:41.699334 23316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260806 02:45:41.699494 23316 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:41.699615 23316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 7 FOLLOWER]: Advancing to term 8
I20260806 02:45:41.704188 23316 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:41.704936 23316 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 election: Requested vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
I20260806 02:45:41.705377 23244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a225e002d607451a94b3686f10aeea9a" candidate_term: 8 candidate_status { last_received { term: 7 index: 110 } } ignore_live_leader: false dest_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22"
I20260806 02:45:41.705612 23244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 7 FOLLOWER]: Advancing to term 8
W20260806 02:45:41.706584 23081 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:41.707978 23244 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a225e002d607451a94b3686f10aeea9a in term 8.
I20260806 02:45:41.708369 23079 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: 894be089543b4aa29b3edfcc5f3085e9
I20260806 02:45:41.708740 23316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 8 FOLLOWER]: Leader election won for term 8
I20260806 02:45:41.709270 23316 raft_consensus.cc:686] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 8 LEADER]: Becoming Leader. State: Replica: a225e002d607451a94b3686f10aeea9a, State: Running, Role: LEADER
I20260806 02:45:41.709815 23316 consensus_queue.cc:237] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 110, Committed index: 110, Last appended: 7.110, Last appended by leader: 110, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:41.712235 23321 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a225e002d607451a94b3686f10aeea9a. Latest consensus state: current_term: 8 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:41.712702 23321 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:41.713521 23323 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:45:41.716979 23323 catalog_manager.cc:669] Loaded metadata for table default.table_e41f56f2e4a1439ca6b8f2bca460a670 [id=1bbd1184e7754c5bb9ffdf82791c73fb]
I20260806 02:45:41.717837 23323 catalog_manager.cc:669] Loaded metadata for table default.table_97ee2606f8aa47dcb2c67a9ae29dd3fb [id=370d5895c35f4c96878270e71cac2443]
I20260806 02:45:41.718441 23323 catalog_manager.cc:669] Loaded metadata for table default.table_ea1a38afd92a4b15bfabb80e58f8c644 [id=383ba93bcba140c1b1bd2b36f8720b86]
I20260806 02:45:41.719013 23323 catalog_manager.cc:669] Loaded metadata for table default.table_d7249617a95049389b90db3bd551ace4 [id=4179f72ea45043e39ddce8e93c3429e9]
I20260806 02:45:41.719439 23323 catalog_manager.cc:669] Loaded metadata for table default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e]
I20260806 02:45:41.719842 23323 catalog_manager.cc:669] Loaded metadata for table default.table_0febb4673dff4e20ae8124efba9627c8 [id=5cb271f99441401cad4fd2e40e943a6e]
I20260806 02:45:41.720188 23323 catalog_manager.cc:669] Loaded metadata for table default.table_21221d53bf9f4c46a5ffe671a1ffe424 [id=610932bdd3f94b4e886d586c28ff6249]
I20260806 02:45:41.720609 23323 catalog_manager.cc:669] Loaded metadata for table default.table_e8df6dde49954df7b57567fc0cd12872 [id=a0cffb03ae7641f8848b8d2aab0fca8b]
I20260806 02:45:41.720965 23323 catalog_manager.cc:669] Loaded metadata for table default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26]
I20260806 02:45:41.721463 23323 catalog_manager.cc:669] Loaded metadata for table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]
I20260806 02:45:41.722000 23323 catalog_manager.cc:669] Loaded metadata for table default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c]
I20260806 02:45:41.722553 23323 catalog_manager.cc:669] Loaded metadata for table default.table_00c06d2b7d6d40658a4390f4ee3bbab5 [id=c6a05485b0a54967ba8904aa969edd3f]
I20260806 02:45:41.723088 23323 catalog_manager.cc:669] Loaded metadata for table default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704]
I20260806 02:45:41.723590 23323 catalog_manager.cc:669] Loaded metadata for table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e]
I20260806 02:45:41.724042 23323 catalog_manager.cc:669] Loaded metadata for table default.table_c5be13d5ad2c4a479d2cc9074d14038f [id=f6939f0d45ae4f358d206a8cf2fe5db0]
I20260806 02:45:41.728125 23323 tablet_loader.cc:96] loaded metadata for tablet 043839a5eca143d5882f40faaab39692 (table default.table_21221d53bf9f4c46a5ffe671a1ffe424 [id=610932bdd3f94b4e886d586c28ff6249])
I20260806 02:45:41.728819 23323 tablet_loader.cc:96] loaded metadata for tablet 10189388393d47ce900b44cbad3ff425 (table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e])
I20260806 02:45:41.729465 23323 tablet_loader.cc:96] loaded metadata for tablet 237439c720344cfd97bd04bf8368a375 (table default.table_ea1a38afd92a4b15bfabb80e58f8c644 [id=383ba93bcba140c1b1bd2b36f8720b86])
I20260806 02:45:41.729815 23323 tablet_loader.cc:96] loaded metadata for tablet 290cc76d8c174a1fa3645f3bfe754b52 (table default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c])
I20260806 02:45:41.730259 23323 tablet_loader.cc:96] loaded metadata for tablet 293b93e6322842aaaa786570f2810548 (table default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26])
I20260806 02:45:41.730911 23323 tablet_loader.cc:96] loaded metadata for tablet 3196f6ee70d34d298b7db56d9759fab7 (table default.table_00c06d2b7d6d40658a4390f4ee3bbab5 [id=c6a05485b0a54967ba8904aa969edd3f])
I20260806 02:45:41.731547 23323 tablet_loader.cc:96] loaded metadata for tablet 3d44c58ba1f044b38f10da10cec410e3 (table default.table_e41f56f2e4a1439ca6b8f2bca460a670 [id=1bbd1184e7754c5bb9ffdf82791c73fb])
I20260806 02:45:41.732199 23323 tablet_loader.cc:96] loaded metadata for tablet 50c46742a5a44d54aba65eec2f36aa09 (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022])
I20260806 02:45:41.732892 23323 tablet_loader.cc:96] loaded metadata for tablet 62b1e9f4216845739fc757c975eb8c02 (table default.table_97ee2606f8aa47dcb2c67a9ae29dd3fb [id=370d5895c35f4c96878270e71cac2443])
I20260806 02:45:41.734074 23323 tablet_loader.cc:96] loaded metadata for tablet 8243a6730a6e4939998608a4e4b63a18 (table default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e])
I20260806 02:45:41.734789 23323 tablet_loader.cc:96] loaded metadata for tablet a6e1420c19fc44b3901f653f137ec6c1 (table default.table_e8df6dde49954df7b57567fc0cd12872 [id=a0cffb03ae7641f8848b8d2aab0fca8b])
I20260806 02:45:41.735940 23323 tablet_loader.cc:96] loaded metadata for tablet bea6ade491c6486393aca875767dc1b3 (table default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704])
I20260806 02:45:41.736070 23059 master_runner.cc:186] Error getting master registration for 127.20.60.60:42309: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111),
I20260806 02:45:41.736276 23059 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:45:41.737074 23323 tablet_loader.cc:96] loaded metadata for tablet cbfd41ea3624457da227733bea68ffc5 (table default.table_0febb4673dff4e20ae8124efba9627c8 [id=5cb271f99441401cad4fd2e40e943a6e])
I20260806 02:45:41.738410 23323 tablet_loader.cc:96] loaded metadata for tablet f9fd42a920b24117a4d687fec13ad17b (table default.table_d7249617a95049389b90db3bd551ace4 [id=4179f72ea45043e39ddce8e93c3429e9])
I20260806 02:45:41.738807 23323 tablet_loader.cc:96] loaded metadata for tablet fdc39b7dea2c43cebc730cffacd38d10 (table default.table_c5be13d5ad2c4a479d2cc9074d14038f [id=f6939f0d45ae4f358d206a8cf2fe5db0])
I20260806 02:45:41.739447 23323 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:45:41.741523 23323 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:41.741603 23323 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:45:41.743952 23323 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:45:41.745661 23323 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Loaded TSK: 0
I20260806 02:45:41.746626 23323 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:45:41.748186 23323 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260806 02:45:41.749863 23323 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: 29
I20260806 02:45:41.871591 23100 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58908:
table { table_name: "default.Table_c453f255605e4419859a21f098c7243e" } modify_external_catalogs: true
I20260806 02:45:41.872020 23100 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58908:
table { table_name: "default.Table_c453f255605e4419859a21f098c7243e" } modify_external_catalogs: true
I20260806 02:45:41.871961 23096 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58908:
table { table_name: "default.Table_0e9bb7fa1ed44ca0af8546b2c255fa62" } modify_external_catalogs: true
I20260806 02:45:41.872329 23096 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58908:
table { table_name: "default.Table_0e9bb7fa1ed44ca0af8546b2c255fa62" } modify_external_catalogs: true
I20260806 02:45:41.873093 23099 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58908:
table { table_name: "default.Table_e8df6dde49954df7b57567fc0cd12872" } modify_external_catalogs: true
I20260806 02:45:41.872779 23093 master_service.cc:946] ReplaceTablet: received request to replace tablet 178955fff8a44eb7b629f09adff0d456 from {username='slave'} at 127.0.0.1:58908
I20260806 02:45:41.874279 23095 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58908:
table { table_name: "default.Table_ea1a38afd92a4b15bfabb80e58f8c644" } modify_external_catalogs: true
I20260806 02:45:41.874636 23095 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58908:
table { table_name: "default.Table_ea1a38afd92a4b15bfabb80e58f8c644" } modify_external_catalogs: true
I20260806 02:45:41.874878 23091 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58908:
table { table_name: "default.Table_00c06d2b7d6d40658a4390f4ee3bbab5" } new_table_name: "default.Table_6fff046fe11443558b808f8516187245" modify_external_catalogs: true
I20260806 02:45:41.872419 23097 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58908:
name: "default.Table_2b3ca9846e244fc593dc07fd72a58f3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:41.875663 23099 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58908:
table { table_name: "default.Table_e8df6dde49954df7b57567fc0cd12872" } modify_external_catalogs: true
I20260806 02:45:41.874074 23087 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58908:
name: "default.Table_8ee8dd77f08a4c9d9e3d4f345a9a6bbf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:41.876546 23093 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58908:
name: "default.Table_8a4c223baf6149efb9e759a304f00edd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:41.877352 23092 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58908:
table { table_name: "default.Table_8797f6f61c444bd0a4e4555f6ca2fe30" } new_table_name: "default.Table_97ee2606f8aa47dcb2c67a9ae29dd3fb" modify_external_catalogs: true
I20260806 02:45:41.877725 23088 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58908:
table { table_name: "default.Table_21221d53bf9f4c46a5ffe671a1ffe424" } new_table_name: "default.Table_fb2ecedbb20f49d2a659b25c01f24887" modify_external_catalogs: true
F20260806 02:45:41.878042 23160 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 31): 31 DROP_TABLE default.table_1ff877d50df44dccb8be0241b847e8c9
*** Check failure stack trace: ***
@ 0x7f86731b49ad google::LogMessage::Fail() at ??:0
@ 0x7f86731b788a google::LogMessage::SendToLog() at ??:0
@ 0x7f86731b44de google::LogMessage::Flush() at ??:0
@ 0x7f86731b46d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f8688d52ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f8688d51901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f8688d5866c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f8688d584e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f8675ddeeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f86790b86db start_thread at ??:0
@ 0x7f867204e61f clone at ??:0
W20260806 02:45:42.105753 21977 connection.cc:535] client connection to 127.20.60.62:33227 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:45:42.107002 21977 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.20.60.62:33227)
W20260806 02:45:42.107353 21977 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.20.60.62:33227)
W20260806 02:45:42.108047 21977 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.20.60.62:33227)
W20260806 02:45:42.108589 21977 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.20.60.62:33227)
I20260806 02:45:42.264703 21762 heartbeater.cc:503] Master 127.20.60.62:33227 was elected leader, sending a full tablet report...
I20260806 02:45:42.314571 21951 heartbeater.cc:503] Master 127.20.60.62:33227 was elected leader, sending a full tablet report...
I20260806 02:45:42.736109 20720 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu pid 23188
I20260806 02:45:42.743595 23266 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 02:45:42.855904 20720 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu with pid 23188
I20260806 02:45:43.277137 23341 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:43.277443 23341 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:43.278156 23341 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:45:43.279194 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:45:43.279399 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:45:43.279587 21803 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:43.279879 23341 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:45:43.386780 20720 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
/tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.61:36797
--webserver_interface=127.20.60.61
--webserver_port=46303
--builtin_ntp_servers=127.20.60.20:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260806 02:45:43.522545 23341 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:43.522775 23341 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:43.523417 23341 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:45:43.524292 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:45:43.524477 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:45:43.524624 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:43.524886 23341 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:45:43.788275 23347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:45:43.788678 23347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:45:43.788841 23347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:45:43.796546 23347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:45:43.796672 23347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:45:43.796756 23347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:45:43.796857 23347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:45:43.806733 23347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.60.20:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.61:36797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.60.61
--webserver_port=46303
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 01:55:54 UTC on ccycloud.quasar-csdhqu.root.comops.site
build id 4569536
ASAN enabled
I20260806 02:45:43.808341 23347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:45:43.810683 23347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:45:43.823467 23353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:43.823388 23354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:45:43.824120 23347 server_base.cc:1034] running on GCE node
W20260806 02:45:43.824275 23356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:45:43.825193 23347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:45:43.826797 23347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:45:43.828102 23347 hybrid_clock.cc:648] HybridClock initialized: now 1785984343828024 us; error 53 us; skew 500 ppm
I20260806 02:45:43.828509 23347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 02:45:43.831050 23347 webserver.cc:460] Webserver started at http://127.20.60.61:46303/ using document root <none> and password file <none>
I20260806 02:45:43.831858 23347 fs_manager.cc:359] Metadata directory not provided
I20260806 02:45:43.832019 23347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:45:43.838071 23347 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 02:45:43.841986 23362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:43.843684 23347 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 02:45:43.843865 23347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data,/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22"
format_stamp: "Formatted at 2026-08-06 02:45:17 on dist-test-slave-bw88"
I20260806 02:45:43.845304 23347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:45:43.867704 23347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:45:43.868666 23347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:45:43.868988 23347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:45:43.890138 23347 rpc_server.cc:307] RPC server started. Bound to: 127.20.60.61:36797
I20260806 02:45:43.890170 23427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.60.61:36797 every 8 connection(s)
I20260806 02:45:43.891726 23347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-1/data/info.pb
I20260806 02:45:43.893148 20720 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu as pid 23347
I20260806 02:45:43.900290 23428 sys_catalog.cc:263] Verifying existing consensus state
I20260806 02:45:43.905982 23428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Bootstrap starting.
I20260806 02:45:43.925130 23428 log.cc:824] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Log is configured to *not* fsync() on all Append() calls
I20260806 02:45:44.010342 23428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Bootstrap replayed 1/1 log segments. Stats: ops{read=109 overwritten=1 applied=106 ignored=0} inserts{seen=68 ignored=0} mutations{seen=92 ignored=0} orphaned_commits=0. Pending: 2 replicates
I20260806 02:45:44.011070 23428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Bootstrap complete.
I20260806 02:45:44.019340 23428 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 8 FOLLOWER]: Replica starting. Triggering 2 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:44.022996 23428 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfdbf4ef60014ea98bbcf9b6a9033f22, State: Initialized, Role: FOLLOWER
I20260806 02:45:44.023696 23428 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 106, Last appended: 6.108, Last appended by leader: 108, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:44.024456 23437 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:44.024809 23437 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:44.025691 23428 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:45:44.039659 23456 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:44.039870 23456 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: loading cluster ID for follower catalog manager: success
I20260806 02:45:44.046761 23456 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: acquiring CA information for follower catalog manager: success
I20260806 02:45:44.051045 23456 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 02:45:44.043218 23347 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:45:44.052567 23347 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:45:44.062247 21583 heartbeater.cc:347] Connected to a master server at 127.20.60.61:36797
I20260806 02:45:44.067147 23381 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" instance_seqno: 1785984319928949) as {username='slave'} at 127.20.60.1:52641; Asking this server to re-register.
I20260806 02:45:44.068274 21583 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:44.070354 23381 ts_manager.cc:194] Registered new tserver with Master: 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:44.252184 21763 heartbeater.cc:347] Connected to a master server at 127.20.60.61:36797
I20260806 02:45:44.253836 23381 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" instance_seqno: 1785984320818216) as {username='slave'} at 127.20.60.2:41405; Asking this server to re-register.
I20260806 02:45:44.254513 21763 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:44.256114 23381 ts_manager.cc:194] Registered new tserver with Master: 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:44.396333 21952 heartbeater.cc:347] Connected to a master server at 127.20.60.61:36797
I20260806 02:45:44.398140 23381 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" instance_seqno: 1785984321670316) as {username='slave'} at 127.20.60.3:51829; Asking this server to re-register.
I20260806 02:45:44.398890 21952 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:44.400629 23381 ts_manager.cc:194] Registered new tserver with Master: 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:44.403017 23437 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:44.403335 23437 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:44.405606 23437 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:44.406735 23365 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:45:44.408841 23366 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
W20260806 02:45:44.409104 23365 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:45:44.409368 23365 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a
I20260806 02:45:44.409817 23437 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260806 02:45:45.084175 23462 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:45.084389 23462 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:45.085251 23462 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:45.088081 23365 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:45:45.088266 23366 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:45.088452 23366 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a
I20260806 02:45:45.088809 23462 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260806 02:45:45.953521 23470 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:45.953740 23470 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:45.954448 23470 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:45.956615 23365 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:45:45.956903 23366 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:45.957072 23366 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a
I20260806 02:45:45.957371 23470 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260806 02:45:47.012252 23477 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:47.012451 23477 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:47.013099 23477 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:47.015338 23365 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:45:47.015525 23366 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:47.015687 23366 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a
I20260806 02:45:47.015957 23477 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260806 02:45:47.208312 20720 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
/tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.62:33227
--webserver_interface=127.20.60.62
--webserver_port=34225
--builtin_ntp_servers=127.20.60.20:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260806 02:45:47.566064 23481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:45:47.566363 23481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:45:47.566442 23481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:45:47.574358 23481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:45:47.574517 23481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:45:47.574615 23481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:45:47.574709 23481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:45:47.584540 23481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.60.20:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.62:33227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.60.62
--webserver_port=34225
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 01:55:54 UTC on ccycloud.quasar-csdhqu.root.comops.site
build id 4569536
ASAN enabled
I20260806 02:45:47.586098 23481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:45:47.587992 23481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:45:47.600682 23491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:47.600870 23486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:47.600804 23487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:45:47.601804 23481 server_base.cc:1034] running on GCE node
I20260806 02:45:47.602669 23481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:45:47.604246 23481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:45:47.605458 23481 hybrid_clock.cc:648] HybridClock initialized: now 1785984347605386 us; error 48 us; skew 500 ppm
I20260806 02:45:47.605813 23481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 02:45:47.612372 23481 webserver.cc:460] Webserver started at http://127.20.60.62:34225/ using document root <none> and password file <none>
I20260806 02:45:47.613328 23481 fs_manager.cc:359] Metadata directory not provided
I20260806 02:45:47.613516 23481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:45:47.616082 23476 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:47.616294 23476 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:47.617044 23476 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:45:47.618372 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:45:47.618599 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:45:47.618747 21803 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:47.619027 23476 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:45:47.619624 23481 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 02:45:47.623127 23497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:47.624495 23481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260806 02:45:47.624681 23481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data,/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
uuid: "a225e002d607451a94b3686f10aeea9a"
format_stamp: "Formatted at 2026-08-06 02:45:17 on dist-test-slave-bw88"
I20260806 02:45:47.625756 23481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:45:47.654177 23481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:45:47.655149 23481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:45:47.655488 23481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:45:47.674247 23481 rpc_server.cc:307] RPC server started. Bound to: 127.20.60.62:33227
I20260806 02:45:47.674268 23569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.60.62:33227 every 8 connection(s)
I20260806 02:45:47.675851 23481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-0/data/info.pb
I20260806 02:45:47.683101 20720 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu as pid 23481
I20260806 02:45:47.686144 23570 sys_catalog.cc:263] Verifying existing consensus state
I20260806 02:45:47.691651 23570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Bootstrap starting.
I20260806 02:45:47.712569 23570 log.cc:824] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Log is configured to *not* fsync() on all Append() calls
I20260806 02:45:47.728114 21582 heartbeater.cc:347] Connected to a master server at 127.20.60.62:33227
I20260806 02:45:47.787267 21762 heartbeater.cc:347] Connected to a master server at 127.20.60.62:33227
I20260806 02:45:47.787523 21762 heartbeater.cc:503] Master 127.20.60.62:33227 was elected leader, sending a full tablet report...
I20260806 02:45:47.801326 23570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Bootstrap replayed 1/1 log segments. Stats: ops{read=111 overwritten=0 applied=110 ignored=0} inserts{seen=70 ignored=0} mutations{seen=97 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260806 02:45:47.802099 23570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Bootstrap complete.
I20260806 02:45:47.810142 23570 raft_consensus.cc:348] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 8 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:47.811314 23570 raft_consensus.cc:729] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: a225e002d607451a94b3686f10aeea9a, State: Initialized, Role: FOLLOWER
I20260806 02:45:47.811910 23570 consensus_queue.cc:260] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 110, Last appended: 8.111, Last appended by leader: 111, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:47.812810 23578 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:47.813133 23578 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:47.813954 23570 sys_catalog.cc:564] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:45:47.832573 23593 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:47.832717 23593 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: loading cluster ID for follower catalog manager: success
I20260806 02:45:47.842937 21951 heartbeater.cc:347] Connected to a master server at 127.20.60.62:33227
I20260806 02:45:47.843173 21951 heartbeater.cc:503] Master 127.20.60.62:33227 was elected leader, sending a full tablet report...
I20260806 02:45:47.847925 23516 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" instance_seqno: 1785984321670316) as {username='slave'} at 127.20.60.3:37799; Asking this server to re-register.
I20260806 02:45:47.849027 21951 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:47.852463 23517 ts_manager.cc:194] Registered new tserver with Master: 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:47.854524 23593 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: acquiring CA information for follower catalog manager: success
I20260806 02:45:47.857633 23593 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 02:45:47.879081 23481 master_runner.cc:186] Error getting master registration for 127.20.60.60:42309: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111),
I20260806 02:45:47.885550 23481 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:45:48.214809 23578 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:48.215152 23578 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:48.217178 23578 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:48.217932 23501 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:48.218040 23394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a225e002d607451a94b3686f10aeea9a" candidate_term: 9 candidate_status { last_received { term: 8 index: 111 } } ignore_live_leader: false dest_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" is_pre_election: true
I20260806 02:45:48.218649 23394 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a225e002d607451a94b3686f10aeea9a in term 8.
I20260806 02:45:48.219439 23498 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22; no voters:
W20260806 02:45:48.219735 23501 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:48.219820 23578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260806 02:45:48.219993 23578 raft_consensus.cc:482] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:48.220121 23578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 8 FOLLOWER]: Advancing to term 9
I20260806 02:45:48.224401 23578 raft_consensus.cc:504] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:48.225095 23578 leader_election.cc:288] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 9 election: Requested vote from peers bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
I20260806 02:45:48.225728 23394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a225e002d607451a94b3686f10aeea9a" candidate_term: 9 candidate_status { last_received { term: 8 index: 111 } } ignore_live_leader: false dest_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22"
I20260806 02:45:48.225999 23394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 8 FOLLOWER]: Advancing to term 9
W20260806 02:45:48.226536 23501 leader_election.cc:334] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
I20260806 02:45:48.229981 23394 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a225e002d607451a94b3686f10aeea9a in term 9.
I20260806 02:45:48.230408 23498 leader_election.cc:302] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: 894be089543b4aa29b3edfcc5f3085e9
I20260806 02:45:48.230859 23578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 9 FOLLOWER]: Leader election won for term 9
I20260806 02:45:48.231307 23578 raft_consensus.cc:686] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [term 9 LEADER]: Becoming Leader. State: Replica: a225e002d607451a94b3686f10aeea9a, State: Running, Role: LEADER
I20260806 02:45:48.231838 23578 consensus_queue.cc:237] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 110, Committed index: 110, Last appended: 8.111, Last appended by leader: 111, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:48.234992 23600 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a225e002d607451a94b3686f10aeea9a. Latest consensus state: current_term: 9 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:48.235296 23516 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" instance_seqno: 1785984319928949) as {username='slave'} at 127.20.60.1:36939; Asking this server to re-register.
I20260806 02:45:48.235460 23600 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:48.236107 21582 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:48.236253 23602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:45:48.238179 23516 ts_manager.cc:194] Registered new tserver with Master: 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:48.239764 23602 catalog_manager.cc:669] Loaded metadata for table default.table_e41f56f2e4a1439ca6b8f2bca460a670 [id=1bbd1184e7754c5bb9ffdf82791c73fb]
I20260806 02:45:48.240433 23602 catalog_manager.cc:669] Loaded metadata for table default.table_97ee2606f8aa47dcb2c67a9ae29dd3fb [id=370d5895c35f4c96878270e71cac2443]
I20260806 02:45:48.240796 23602 catalog_manager.cc:669] Loaded metadata for table default.table_ea1a38afd92a4b15bfabb80e58f8c644 [id=383ba93bcba140c1b1bd2b36f8720b86]
I20260806 02:45:48.241137 23602 catalog_manager.cc:669] Loaded metadata for table default.table_d7249617a95049389b90db3bd551ace4 [id=4179f72ea45043e39ddce8e93c3429e9]
I20260806 02:45:48.241592 23602 catalog_manager.cc:669] Loaded metadata for table default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e]
I20260806 02:45:48.241953 23602 catalog_manager.cc:669] Loaded metadata for table default.table_0febb4673dff4e20ae8124efba9627c8 [id=5cb271f99441401cad4fd2e40e943a6e]
I20260806 02:45:48.242278 23602 catalog_manager.cc:669] Loaded metadata for table default.table_21221d53bf9f4c46a5ffe671a1ffe424 [id=610932bdd3f94b4e886d586c28ff6249]
I20260806 02:45:48.242627 23602 catalog_manager.cc:669] Loaded metadata for table default.table_e8df6dde49954df7b57567fc0cd12872 [id=a0cffb03ae7641f8848b8d2aab0fca8b]
I20260806 02:45:48.242954 23602 catalog_manager.cc:669] Loaded metadata for table default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26]
I20260806 02:45:48.243279 23602 catalog_manager.cc:669] Loaded metadata for table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]
I20260806 02:45:48.243611 23602 catalog_manager.cc:669] Loaded metadata for table default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c]
I20260806 02:45:48.243973 23602 catalog_manager.cc:669] Loaded metadata for table default.table_00c06d2b7d6d40658a4390f4ee3bbab5 [id=c6a05485b0a54967ba8904aa969edd3f]
I20260806 02:45:48.244410 23602 catalog_manager.cc:669] Loaded metadata for table default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704]
I20260806 02:45:48.244773 23602 catalog_manager.cc:669] Loaded metadata for table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e]
I20260806 02:45:48.245087 23602 catalog_manager.cc:669] Loaded metadata for table default.table_c5be13d5ad2c4a479d2cc9074d14038f [id=f6939f0d45ae4f358d206a8cf2fe5db0]
I20260806 02:45:48.247977 23602 tablet_loader.cc:96] loaded metadata for tablet 043839a5eca143d5882f40faaab39692 (table default.table_21221d53bf9f4c46a5ffe671a1ffe424 [id=610932bdd3f94b4e886d586c28ff6249])
I20260806 02:45:48.248507 23602 tablet_loader.cc:96] loaded metadata for tablet 10189388393d47ce900b44cbad3ff425 (table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e])
I20260806 02:45:48.248950 23602 tablet_loader.cc:96] loaded metadata for tablet 237439c720344cfd97bd04bf8368a375 (table default.table_ea1a38afd92a4b15bfabb80e58f8c644 [id=383ba93bcba140c1b1bd2b36f8720b86])
I20260806 02:45:48.249253 23602 tablet_loader.cc:96] loaded metadata for tablet 290cc76d8c174a1fa3645f3bfe754b52 (table default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c])
I20260806 02:45:48.249598 23602 tablet_loader.cc:96] loaded metadata for tablet 293b93e6322842aaaa786570f2810548 (table default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26])
I20260806 02:45:48.250072 23602 tablet_loader.cc:96] loaded metadata for tablet 3196f6ee70d34d298b7db56d9759fab7 (table default.table_00c06d2b7d6d40658a4390f4ee3bbab5 [id=c6a05485b0a54967ba8904aa969edd3f])
I20260806 02:45:48.250608 23602 tablet_loader.cc:96] loaded metadata for tablet 3d44c58ba1f044b38f10da10cec410e3 (table default.table_e41f56f2e4a1439ca6b8f2bca460a670 [id=1bbd1184e7754c5bb9ffdf82791c73fb])
I20260806 02:45:48.251065 23602 tablet_loader.cc:96] loaded metadata for tablet 50c46742a5a44d54aba65eec2f36aa09 (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022])
I20260806 02:45:48.251735 23602 tablet_loader.cc:96] loaded metadata for tablet 62b1e9f4216845739fc757c975eb8c02 (table default.table_97ee2606f8aa47dcb2c67a9ae29dd3fb [id=370d5895c35f4c96878270e71cac2443])
I20260806 02:45:48.252519 23602 tablet_loader.cc:96] loaded metadata for tablet 8243a6730a6e4939998608a4e4b63a18 (table default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e])
I20260806 02:45:48.252982 23602 tablet_loader.cc:96] loaded metadata for tablet a6e1420c19fc44b3901f653f137ec6c1 (table default.table_e8df6dde49954df7b57567fc0cd12872 [id=a0cffb03ae7641f8848b8d2aab0fca8b])
I20260806 02:45:48.253710 23602 tablet_loader.cc:96] loaded metadata for tablet bea6ade491c6486393aca875767dc1b3 (table default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704])
I20260806 02:45:48.254398 23602 tablet_loader.cc:96] loaded metadata for tablet cbfd41ea3624457da227733bea68ffc5 (table default.table_0febb4673dff4e20ae8124efba9627c8 [id=5cb271f99441401cad4fd2e40e943a6e])
I20260806 02:45:48.255177 23602 tablet_loader.cc:96] loaded metadata for tablet f9fd42a920b24117a4d687fec13ad17b (table default.table_d7249617a95049389b90db3bd551ace4 [id=4179f72ea45043e39ddce8e93c3429e9])
I20260806 02:45:48.255525 23602 tablet_loader.cc:96] loaded metadata for tablet fdc39b7dea2c43cebc730cffacd38d10 (table default.table_c5be13d5ad2c4a479d2cc9074d14038f [id=f6939f0d45ae4f358d206a8cf2fe5db0])
I20260806 02:45:48.256000 23602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:45:48.257664 23602 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:48.257762 23602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:45:48.259603 23602 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:45:48.261027 23602 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Loaded TSK: 0
I20260806 02:45:48.262024 23602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:45:48.263418 23602 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260806 02:45:48.264894 23602 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: 29
I20260806 02:45:48.291587 21762 heartbeater.cc:503] Master 127.20.60.62:33227 was elected leader, sending a full tablet report...
I20260806 02:45:48.293921 23516 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" instance_seqno: 1785984320818216) as {username='slave'} at 127.20.60.2:42903; Asking this server to re-register.
I20260806 02:45:48.294834 21762 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:48.295087 21762 heartbeater.cc:511] Master 127.20.60.62:33227 requested a full tablet report, sending...
I20260806 02:45:48.297118 23516 ts_manager.cc:194] Registered new tserver with Master: 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:48.299178 23516 catalog_manager.cc:5495] T 10189388393d47ce900b44cbad3ff425 P 88dc3ab284a64961b1fbdb5c53d1267f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3). New cstate: current_term: 1 leader_uuid: "8ebd96b903b944f3b06851324b737bd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } }
I20260806 02:45:48.300617 23516 catalog_manager.cc:5587] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655) does not have the latest schema for tablet 50c46742a5a44d54aba65eec2f36aa09 (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]). Expected version 1 got 0
I20260806 02:45:48.300872 23516 catalog_manager.cc:5495] T 293b93e6322842aaaa786570f2810548 P 88dc3ab284a64961b1fbdb5c53d1267f reported cstate change: term changed from 0 to 1, leader changed from <none> to 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2). New cstate: current_term: 1 leader_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } health_report { overall_health: HEALTHY } } }
I20260806 02:45:48.306937 23394 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 9 FOLLOWER]: Refusing update from remote peer a225e002d607451a94b3686f10aeea9a: Log matching property violated. Preceding OpId in replica: term: 6 index: 108. Preceding OpId from leader: term: 9 index: 113. (index mismatch)
W20260806 02:45:48.307308 23501 consensus_peers.cc:487] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a -> Peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Couldn't send request to peer 894be089543b4aa29b3edfcc5f3085e9. Status: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 02:45:48.307842 23600 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 112, Last known committed idx: 106, Time since last communication: 0.000s
I20260806 02:45:48.314462 23611 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:48.314916 23611 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:48.315483 23605 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a225e002d607451a94b3686f10aeea9a. Latest consensus state: current_term: 9 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:48.315925 23605 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:48.317191 23604 mvcc.cc:205] Tried to move back new op lower bound from 7315391890656206848 to 7315391890361221120. Current Snapshot: MvccSnapshot[applied={T|T < 7315391890656206848}]
I20260806 02:45:48.317561 23605 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:48.317807 23605 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:48.318075 23605 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:48.318339 23605 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:48.322304 23600 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:48.322603 23600 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:48.322973 23578 sys_catalog.cc:455] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 9 leader_uuid: "a225e002d607451a94b3686f10aeea9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:48.323235 23578 sys_catalog.cc:458] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:48.328490 23516 catalog_manager.cc:6040] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: default.table_97ee2606f8aa47dcb2c67a9ae29dd3fb [id=370d5895c35f4c96878270e71cac2443] alter complete (version 2)
I20260806 02:45:48.336674 21681 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cc50b1bfe414464b1863c4e26f295fb with delete_type TABLET_DATA_DELETED (replaced by tablet 290cc76d8c174a1fa3645f3bfe754b52) from {username='slave'} at 127.0.0.1:56520
I20260806 02:45:48.336701 21683 tablet_service.cc:1514] Processing DeleteTablet for tablet 6061f59aedec4e0a9b96239ebe6fa0df with delete_type TABLET_DATA_DELETED (replaced by tablet 8243a6730a6e4939998608a4e4b63a18) from {username='slave'} at 127.0.0.1:56520
I20260806 02:45:48.336755 21682 tablet_service.cc:1514] Processing DeleteTablet for tablet ba32fbee948d4adf96164d36673a1db6 with delete_type TABLET_DATA_DELETED (replaced by tablet cbfd41ea3624457da227733bea68ffc5) from {username='slave'} at 127.0.0.1:56520
I20260806 02:45:48.337495 23616 tablet_replica.cc:330] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:48.337805 23616 raft_consensus.cc:2229] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:45:48.338249 23616 raft_consensus.cc:2258] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:48.338969 23616 ts_tablet_manager.cc:1905] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:48.347581 23616 ts_tablet_manager.cc:1918] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:48.347739 23616 log.cc:1196] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:45:48.348201 23616 ts_tablet_manager.cc:1939] T 6cc50b1bfe414464b1863c4e26f295fb P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:48.349104 23616 tablet_replica.cc:330] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:48.349354 23616 raft_consensus.cc:2229] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:48.349360 23501 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet 6cc50b1bfe414464b1863c4e26f295fb (table default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c]) successfully deleted
I20260806 02:45:48.349570 23616 raft_consensus.cc:2258] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:48.350117 23616 ts_tablet_manager.cc:1905] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:48.355194 23616 ts_tablet_manager.cc:1918] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:45:48.355357 23616 log.cc:1196] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/6061f59aedec4e0a9b96239ebe6fa0df
I20260806 02:45:48.356506 23616 ts_tablet_manager.cc:1939] T 6061f59aedec4e0a9b96239ebe6fa0df P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:48.357609 23626 tablet.cc:1722] T 50c46742a5a44d54aba65eec2f36aa09 P 88dc3ab284a64961b1fbdb5c53d1267f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:48.358986 23616 tablet_replica.cc:330] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:48.359118 23501 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet 6061f59aedec4e0a9b96239ebe6fa0df (table default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e]) successfully deleted
I20260806 02:45:48.359210 23616 raft_consensus.cc:2229] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:45:48.359613 23616 raft_consensus.cc:2258] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:48.359498 23627 tablet.cc:1722] T 50c46742a5a44d54aba65eec2f36aa09 P 48b0c1e2106b4d9b941929ddc6d97dd5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:48.360394 23616 ts_tablet_manager.cc:1905] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:48.369458 23628 tablet.cc:1722] T 50c46742a5a44d54aba65eec2f36aa09 P 8ebd96b903b944f3b06851324b737bd5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 02:45:48.370532 23616 ts_tablet_manager.cc:1918] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:48.370692 23616 log.cc:1196] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/ba32fbee948d4adf96164d36673a1db6
I20260806 02:45:48.371160 23616 ts_tablet_manager.cc:1939] T ba32fbee948d4adf96164d36673a1db6 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:48.372869 23501 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet ba32fbee948d4adf96164d36673a1db6 (table default.table_0febb4673dff4e20ae8124efba9627c8 [id=5cb271f99441401cad4fd2e40e943a6e]) successfully deleted
I20260806 02:45:48.374809 23506 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47352:
name: "default.Table_6d994a6895c3485cb404565731c2e7fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:48.375124 23509 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47352:
name: "default.Table_5ed544635d9c480cb8c61f25b7f0f84f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:48.375886 23507 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47352:
table { table_name: "default.Table_e24bd493305d46a4977ed544ce47cc83" } new_table_name: "default.Table_8e31b0866b344f85b9631a2e31b68af1" modify_external_catalogs: true
I20260806 02:45:48.376434 23508 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47352:
table { table_name: "default.Table_ddb47e155c91465f8e38147105f2a6bd" } new_table_name: "default.Table_a7570be8f5c74682ac8df015be8417b6" modify_external_catalogs: true
I20260806 02:45:48.376775 23510 master_service.cc:946] ReplaceTablet: received request to replace tablet ba32fbee948d4adf96164d36673a1db6 from {username='slave'} at 127.0.0.1:47352
I20260806 02:45:48.377473 21951 heartbeater.cc:503] Master 127.20.60.62:33227 was elected leader, sending a full tablet report...
I20260806 02:45:48.377689 23510 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47352:
table { table_name: "default.Table_ea1a38afd92a4b15bfabb80e58f8c644" } modify_external_catalogs: true
I20260806 02:45:48.377924 23510 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47352:
table { table_name: "default.Table_ea1a38afd92a4b15bfabb80e58f8c644" } modify_external_catalogs: true
I20260806 02:45:48.378394 23516 master_service.cc:946] ReplaceTablet: received request to replace tablet bea6ade491c6486393aca875767dc1b3 from {username='slave'} at 127.0.0.1:47352
I20260806 02:45:48.379699 23515 master_service.cc:946] ReplaceTablet: received request to replace tablet 6061f59aedec4e0a9b96239ebe6fa0df from {username='slave'} at 127.0.0.1:47352
I20260806 02:45:48.379976 23517 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47352:
name: "default.Table_e412d99677f440648e5e0fea418116a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:48.381525 23511 master_service.cc:946] ReplaceTablet: received request to replace tablet 6cc50b1bfe414464b1863c4e26f295fb from {username='slave'} at 127.0.0.1:47352
I20260806 02:45:48.382210 23511 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47352:
table { table_name: "default.Table_0e9bb7fa1ed44ca0af8546b2c255fa62" } modify_external_catalogs: true
I20260806 02:45:48.382503 23511 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47352:
table { table_name: "default.Table_0e9bb7fa1ed44ca0af8546b2c255fa62" } modify_external_catalogs: true
I20260806 02:45:48.382714 23513 master_service.cc:946] ReplaceTablet: received request to replace tablet ba32fbee948d4adf96164d36673a1db6 from {username='slave'} at 127.0.0.1:47352
I20260806 02:45:48.387583 23516 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a225e002d607451a94b3686f10aeea9a: Sending DeleteTablet for 3 replicas of tablet bea6ade491c6486393aca875767dc1b3
I20260806 02:45:48.387955 23515 catalog_manager.cc:5587] TS 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883) does not have the latest schema for tablet 8243a6730a6e4939998608a4e4b63a18 (table default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e]). Expected version 1 got 0
I20260806 02:45:48.388497 21680 tablet_service.cc:1514] Processing DeleteTablet for tablet bea6ade491c6486393aca875767dc1b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 7839cfd0a7fa44d3b6fefdfde3d9442b) from {username='slave'} at 127.0.0.1:56520
I20260806 02:45:48.388789 23516 catalog_manager.cc:6565] ReplaceTablet: tablet bea6ade491c6486393aca875767dc1b3 deleted and replaced by tablet 7839cfd0a7fa44d3b6fefdfde3d9442b
I20260806 02:45:48.389300 23516 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47352:
name: "default.Table_2b3ca9846e244fc593dc07fd72a58f3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260806 02:45:48.389734 23591 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 31): 31 DROP_TABLE default.table_1ff877d50df44dccb8be0241b847e8c9
*** Check failure stack trace: ***
W20260806 02:45:48.391630 23515 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 8243a6730a6e4939998608a4e4b63a18 (table default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 8243a6730a6e4939998608a4e4b63a18 consensus state has no leader
I20260806 02:45:48.392491 23515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47352:
table { table_name: "default.Table_c453f255605e4419859a21f098c7243e" } modify_external_catalogs: true
I20260806 02:45:48.393477 23616 tablet_replica.cc:330] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f: stopping tablet replica
I20260806 02:45:48.393751 23616 raft_consensus.cc:2229] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:48.393990 23616 raft_consensus.cc:2258] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:48.394670 23616 ts_tablet_manager.cc:1905] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:48.394412 23513 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47352:
table { table_name: "default.Table_e8df6dde49954df7b57567fc0cd12872" } modify_external_catalogs: true
@ 0x7fc2593f69ad google::LogMessage::Fail() at ??:0
@ 0x7fc2593f988a google::LogMessage::SendToLog() at ??:0
@ 0x7fc2593f64de google::LogMessage::Flush() at ??:0
@ 0x7fc2593f66d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7fc26ef94ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260806 02:45:48.408677 23513 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47352:
table { table_name: "default.Table_e8df6dde49954df7b57567fc0cd12872" } modify_external_catalogs: true
I20260806 02:45:48.409020 23515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47352:
table { table_name: "default.Table_c453f255605e4419859a21f098c7243e" } modify_external_catalogs: true
I20260806 02:45:48.410553 23616 ts_tablet_manager.cc:1918] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:48.410732 23616 log.cc:1196] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-1/wal/wals/bea6ade491c6486393aca875767dc1b3
I20260806 02:45:48.411249 23616 ts_tablet_manager.cc:1939] T bea6ade491c6486393aca875767dc1b3 P 88dc3ab284a64961b1fbdb5c53d1267f: Deleting consensus metadata
I20260806 02:45:48.411592 21680 tablet_service.cc:1467] Processing CreateTablet for tablet 7839cfd0a7fa44d3b6fefdfde3d9442b (DEFAULT_TABLE table=default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:48.412153 21680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7839cfd0a7fa44d3b6fefdfde3d9442b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:48.412649 23501 catalog_manager.cc:4841] TS 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): tablet bea6ade491c6486393aca875767dc1b3 (table default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704]) successfully deleted
@ 0x7fc26ef93901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260806 02:45:48.417953 21862 tablet_service.cc:1514] Processing DeleteTablet for tablet bea6ade491c6486393aca875767dc1b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 7839cfd0a7fa44d3b6fefdfde3d9442b) from {username='slave'} at 127.0.0.1:38274
I20260806 02:45:48.419528 21865 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cc50b1bfe414464b1863c4e26f295fb with delete_type TABLET_DATA_DELETED (replaced by tablet 290cc76d8c174a1fa3645f3bfe754b52) from {username='slave'} at 127.0.0.1:38274
I20260806 02:45:48.420717 21865 tablet_service.cc:1514] Processing DeleteTablet for tablet 6061f59aedec4e0a9b96239ebe6fa0df with delete_type TABLET_DATA_DELETED (replaced by tablet 8243a6730a6e4939998608a4e4b63a18) from {username='slave'} at 127.0.0.1:38274
I20260806 02:45:48.421272 21865 tablet_service.cc:1514] Processing DeleteTablet for tablet ba32fbee948d4adf96164d36673a1db6 with delete_type TABLET_DATA_DELETED (replaced by tablet cbfd41ea3624457da227733bea68ffc5) from {username='slave'} at 127.0.0.1:38274
I20260806 02:45:48.422032 21865 tablet_service.cc:1467] Processing CreateTablet for tablet 7839cfd0a7fa44d3b6fefdfde3d9442b (DEFAULT_TABLE table=default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:48.422557 21865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7839cfd0a7fa44d3b6fefdfde3d9442b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:45:48.423166 21498 tablet_service.cc:1514] Processing DeleteTablet for tablet bea6ade491c6486393aca875767dc1b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 7839cfd0a7fa44d3b6fefdfde3d9442b) from {username='slave'} at 127.0.0.1:49744
I20260806 02:45:48.423444 21495 tablet_service.cc:1467] Processing CreateTablet for tablet 7839cfd0a7fa44d3b6fefdfde3d9442b (DEFAULT_TABLE table=default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:45:48.423889 21495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7839cfd0a7fa44d3b6fefdfde3d9442b. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fc26ef9a66c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
W20260806 02:45:48.426784 23500 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 8243a6730a6e4939998608a4e4b63a18 (table default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 8243a6730a6e4939998608a4e4b63a18 consensus state has no leader
I20260806 02:45:48.433559 23636 tablet_replica.cc:330] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: stopping tablet replica
I20260806 02:45:48.433837 23636 raft_consensus.cc:2229] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:45:48.434358 23636 raft_consensus.cc:2258] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:48.435173 23636 ts_tablet_manager.cc:1905] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7fc26ef9a4e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260806 02:45:48.441227 23634 tablet_replica.cc:330] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:48.441532 23634 raft_consensus.cc:2229] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:48.442235 23634 raft_consensus.cc:2258] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260806 02:45:48.443154 23634 ts_tablet_manager.cc:1905] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7fc25c020eaf kudu::Thread::SuperviseThread() at ??:0
I20260806 02:45:48.448544 23637 tablet_bootstrap.cc:492] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5: Bootstrap starting.
@ 0x7fc25f2fa6db start_thread at ??:0
I20260806 02:45:48.451058 23637 tablet_bootstrap.cc:654] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5: Neither blocks nor log segments found. Creating new log.
@ 0x7fc25829061f clone at ??:0
I20260806 02:45:48.456274 23638 tablet_bootstrap.cc:492] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 88dc3ab284a64961b1fbdb5c53d1267f: Bootstrap starting.
I20260806 02:45:48.459678 23640 tablet_bootstrap.cc:492] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 48b0c1e2106b4d9b941929ddc6d97dd5: Bootstrap starting.
I20260806 02:45:48.462010 23634 ts_tablet_manager.cc:1918] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:48.462054 23640 tablet_bootstrap.cc:654] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 48b0c1e2106b4d9b941929ddc6d97dd5: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:48.462164 23634 log.cc:1196] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/bea6ade491c6486393aca875767dc1b3
I20260806 02:45:48.469094 23636 ts_tablet_manager.cc:1918] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:48.469797 23638 tablet_bootstrap.cc:654] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 88dc3ab284a64961b1fbdb5c53d1267f: Neither blocks nor log segments found. Creating new log.
I20260806 02:45:48.470798 23640 tablet_bootstrap.cc:492] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 48b0c1e2106b4d9b941929ddc6d97dd5: No bootstrap required, opened a new log
I20260806 02:45:48.470971 23640 ts_tablet_manager.cc:1397] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent bootstrapping tablet: real 0.011s user 0.011s sys 0.000s
I20260806 02:45:48.471498 23634 ts_tablet_manager.cc:1939] T bea6ade491c6486393aca875767dc1b3 P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:48.472049 23640 raft_consensus.cc:348] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:48.472405 23640 raft_consensus.cc:374] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:48.472527 23640 raft_consensus.cc:729] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b0c1e2106b4d9b941929ddc6d97dd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:48.472689 23634 tablet_replica.cc:330] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:48.472927 23634 raft_consensus.cc:2229] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:48.472844 23640 consensus_queue.cc:260] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 48b0c1e2106b4d9b941929ddc6d97dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:48.473119 23634 raft_consensus.cc:2258] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:48.473728 21582 heartbeater.cc:503] Master 127.20.60.62:33227 was elected leader, sending a full tablet report...
I20260806 02:45:48.473775 23634 ts_tablet_manager.cc:1905] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:48.478112 23640 ts_tablet_manager.cc:1428] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 48b0c1e2106b4d9b941929ddc6d97dd5: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260806 02:45:48.479893 23637 tablet_bootstrap.cc:492] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5: No bootstrap required, opened a new log
I20260806 02:45:48.469287 23636 log.cc:1196] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-0/wal/wals/bea6ade491c6486393aca875767dc1b3
I20260806 02:45:48.480048 23637 ts_tablet_manager.cc:1397] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5: Time spent bootstrapping tablet: real 0.032s user 0.005s sys 0.000s
I20260806 02:45:48.480472 23636 ts_tablet_manager.cc:1939] T bea6ade491c6486393aca875767dc1b3 P 48b0c1e2106b4d9b941929ddc6d97dd5: Deleting consensus metadata
I20260806 02:45:48.480963 23637 raft_consensus.cc:348] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:48.481308 23637 raft_consensus.cc:374] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:48.481642 23637 raft_consensus.cc:729] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Initialized, Role: FOLLOWER
I20260806 02:45:48.481951 23637 consensus_queue.cc:260] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:48.484045 23638 tablet_bootstrap.cc:492] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 88dc3ab284a64961b1fbdb5c53d1267f: No bootstrap required, opened a new log
I20260806 02:45:48.484215 23638 ts_tablet_manager.cc:1397] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent bootstrapping tablet: real 0.028s user 0.006s sys 0.000s
I20260806 02:45:48.485222 23634 ts_tablet_manager.cc:1918] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:48.485118 23638 raft_consensus.cc:348] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:48.485417 23638 raft_consensus.cc:374] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:45:48.485428 23634 log.cc:1196] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:45:48.485540 23638 raft_consensus.cc:729] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88dc3ab284a64961b1fbdb5c53d1267f, State: Initialized, Role: FOLLOWER
I20260806 02:45:48.485900 23634 ts_tablet_manager.cc:1939] T 6cc50b1bfe414464b1863c4e26f295fb P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:48.485788 23638 consensus_queue.cc:260] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 88dc3ab284a64961b1fbdb5c53d1267f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:48.486593 23638 ts_tablet_manager.cc:1428] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 88dc3ab284a64961b1fbdb5c53d1267f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 02:45:48.487772 23634 tablet_replica.cc:330] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:48.487972 23634 raft_consensus.cc:2229] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:48.488180 23634 raft_consensus.cc:2258] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:48.488765 23634 ts_tablet_manager.cc:1905] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:48.493559 23637 ts_tablet_manager.cc:1428] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5: Time spent starting tablet: real 0.013s user 0.000s sys 0.003s
I20260806 02:45:48.495400 23634 ts_tablet_manager.cc:1918] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:45:48.495546 23634 log.cc:1196] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/6061f59aedec4e0a9b96239ebe6fa0df
I20260806 02:45:48.496017 23634 ts_tablet_manager.cc:1939] T 6061f59aedec4e0a9b96239ebe6fa0df P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
I20260806 02:45:48.496973 23634 tablet_replica.cc:330] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5: stopping tablet replica
I20260806 02:45:48.497179 23634 raft_consensus.cc:2229] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:45:48.497376 23634 raft_consensus.cc:2258] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:45:48.497987 23634 ts_tablet_manager.cc:1905] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:45:48.504431 23634 ts_tablet_manager.cc:1918] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 02:45:48.504597 23634 log.cc:1196] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5: Deleting WAL directory at /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/ts-2/wal/wals/ba32fbee948d4adf96164d36673a1db6
I20260806 02:45:48.505038 23634 ts_tablet_manager.cc:1939] T ba32fbee948d4adf96164d36673a1db6 P 8ebd96b903b944f3b06851324b737bd5: Deleting consensus metadata
W20260806 02:45:48.505002 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 1: this message will repeat every 5th retry.
W20260806 02:45:48.717525 21618 connection.cc:535] server connection from 127.0.0.1:56520 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:45:48.717512 21435 connection.cc:535] server connection from 127.0.0.1:49744 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:45:48.717814 21435 connection.cc:535] client connection to 127.20.60.62:33227 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:45:48.717819 21618 connection.cc:535] client connection to 127.20.60.62:33227 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:45:48.718412 21977 connection.cc:535] client connection to 127.20.60.62:33227 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:45:48.718580 21804 connection.cc:535] server connection from 127.0.0.1:38274 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:45:48.718796 21804 connection.cc:535] client connection to 127.20.60.62:33227 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:45:48.719164 21977 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.20.60.62:33227)
W20260806 02:45:48.719919 21977 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.20.60.62:33227)
W20260806 02:45:48.720191 21977 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.20.60.62:33227)
W20260806 02:45:48.720963 21977 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.20.60.62:33227)
I20260806 02:45:48.743513 23476 raft_consensus.cc:482] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:48.743708 23476 raft_consensus.cc:504] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:48.744400 23476 leader_election.cc:288] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:48.744892 21705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7839cfd0a7fa44d3b6fefdfde3d9442b" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" is_pre_election: true
I20260806 02:45:48.744961 21519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7839cfd0a7fa44d3b6fefdfde3d9442b" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" is_pre_election: true
I20260806 02:45:48.745195 21705 raft_consensus.cc:2454] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:48.745230 21519 raft_consensus.cc:2454] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 0.
I20260806 02:45:48.745703 21806 leader_election.cc:302] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:48.745992 23476 raft_consensus.cc:2781] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:45:48.746140 23476 raft_consensus.cc:482] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:45:48.746246 23476 raft_consensus.cc:3037] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:48.749872 23476 raft_consensus.cc:504] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:48.750687 23476 leader_election.cc:288] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Requested vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
I20260806 02:45:48.751232 21705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7839cfd0a7fa44d3b6fefdfde3d9442b" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88dc3ab284a64961b1fbdb5c53d1267f"
I20260806 02:45:48.751526 21705 raft_consensus.cc:3037] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 88dc3ab284a64961b1fbdb5c53d1267f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:48.753631 21519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7839cfd0a7fa44d3b6fefdfde3d9442b" candidate_uuid: "8ebd96b903b944f3b06851324b737bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5"
I20260806 02:45:48.753891 21519 raft_consensus.cc:3037] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:45:48.754736 21705 raft_consensus.cc:2454] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 88dc3ab284a64961b1fbdb5c53d1267f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:48.755213 21806 leader_election.cc:302] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5; no voters:
I20260806 02:45:48.755539 23476 raft_consensus.cc:2781] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:45:48.757042 21519 raft_consensus.cc:2454] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ebd96b903b944f3b06851324b737bd5 in term 1.
I20260806 02:45:48.758128 23476 raft_consensus.cc:686] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [term 1 LEADER]: Becoming Leader. State: Replica: 8ebd96b903b944f3b06851324b737bd5, State: Running, Role: LEADER
I20260806 02:45:48.758490 23476 consensus_queue.cc:237] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:48.910267 23659 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a225e002d607451a94b3686f10aeea9a)
I20260806 02:45:48.910490 23659 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:48.911345 23659 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:48.913647 23366 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
W20260806 02:45:48.913904 23365 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:45:48.914033 23365 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a
I20260806 02:45:48.914314 23659 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260806 02:45:49.053946 23347 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:45:49.054139 23347 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:45:49.064481 23476 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:49.064710 23476 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:49.065428 23476 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:45:49.066390 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:45:49.066568 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:45:49.066749 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:49.067047 23476 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:45:49.125057 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 1: this message will repeat every 5th retry.
I20260806 02:45:49.501814 23665 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a225e002d607451a94b3686f10aeea9a)
I20260806 02:45:49.502051 23665 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:49.502877 23665 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:49.503513 23366 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260806 02:45:49.505298 23366 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
W20260806 02:45:49.505713 23365 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:45:49.505821 23365 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a
I20260806 02:45:49.506140 23665 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260806 02:45:49.651872 23476 consensus_queue.cc:1035] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:49.660096 23671 consensus_queue.cc:1035] T 7839cfd0a7fa44d3b6fefdfde3d9442b P 8ebd96b903b944f3b06851324b737bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:45:50.025765 23683 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a225e002d607451a94b3686f10aeea9a)
I20260806 02:45:50.025981 23683 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:50.026777 23683 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
W20260806 02:45:50.029294 23366 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309): Network error: Client connection negotiation failed: client connection to 127.20.60.60:42309: connect: Connection refused (error 111)
W20260806 02:45:50.029505 23365 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:45:50.029627 23365 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: 894be089543b4aa29b3edfcc5f3085e9, a225e002d607451a94b3686f10aeea9a
I20260806 02:45:50.029917 23683 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260806 02:45:50.192974 23671 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:50.193269 23671 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:50.194021 23671 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:45:50.195150 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:45:50.195374 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:45:50.195523 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:50.195835 23671 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:45:50.719283 23693 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:45:50.719527 23693 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:50.720317 23693 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:45:50.721362 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:45:50.721551 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:45:50.721742 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:45:50.721992 23693 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:45:50.772835 23693 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:45:50.773038 23693 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:50.773803 23693 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:45:50.774677 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:45:50.774863 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:45:50.775149 21436 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:45:50.775456 23693 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:45:50.975080 20720 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
/tmp/dist-test-taskYdWHu7/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.60:42309
--webserver_interface=127.20.60.60
--webserver_port=46389
--builtin_ntp_servers=127.20.60.20:44373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260806 02:45:51.345965 23701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:45:51.346269 23701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:45:51.346415 23701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:45:51.354573 23701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:45:51.354691 23701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:45:51.354787 23701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:45:51.354902 23701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:45:51.364588 23701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.60.20:44373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:39385
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.60.60:42309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.60.60
--webserver_port=46389
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 01:55:54 UTC on ccycloud.quasar-csdhqu.root.comops.site
build id 4569536
ASAN enabled
I20260806 02:45:51.366101 23701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:45:51.367887 23701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:45:51.380561 23718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:51.380647 23717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:45:51.380616 23720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:45:51.381413 23701 server_base.cc:1034] running on GCE node
I20260806 02:45:51.382345 23701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:45:51.384092 23701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:45:51.385393 23701 hybrid_clock.cc:648] HybridClock initialized: now 1785984351385283 us; error 67 us; skew 500 ppm
I20260806 02:45:51.385759 23701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 02:45:51.388150 23701 webserver.cc:460] Webserver started at http://127.20.60.60:46389/ using document root <none> and password file <none>
I20260806 02:45:51.388890 23701 fs_manager.cc:359] Metadata directory not provided
I20260806 02:45:51.389057 23701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:45:51.394596 23701 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 02:45:51.397783 23728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:45:51.399333 23701 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 02:45:51.399523 23701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data,/tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/wal
uuid: "894be089543b4aa29b3edfcc5f3085e9"
format_stamp: "Formatted at 2026-08-06 02:45:18 on dist-test-slave-bw88"
I20260806 02:45:51.400517 23701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:45:51.424506 23701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:45:51.425597 23701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:45:51.425951 23701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:45:51.447309 23701 rpc_server.cc:307] RPC server started. Bound to: 127.20.60.60:42309
I20260806 02:45:51.447330 23796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.60.60:42309 every 8 connection(s)
I20260806 02:45:51.448920 23701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYdWHu7/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1785984301749593-20720-0/minicluster-data/master-2/data/info.pb
I20260806 02:45:51.449895 20720 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYdWHu7/build/asan/bin/kudu as pid 23701
I20260806 02:45:51.457041 23797 sys_catalog.cc:263] Verifying existing consensus state
I20260806 02:45:51.460834 23797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Bootstrap starting.
I20260806 02:45:51.479212 23797 log.cc:824] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Log is configured to *not* fsync() on all Append() calls
I20260806 02:45:51.536832 21581 heartbeater.cc:347] Connected to a master server at 127.20.60.60:42309
I20260806 02:45:51.569561 23797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=110 overwritten=0 applied=110 ignored=0} inserts{seen=70 ignored=0} mutations{seen=97 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 02:45:51.570106 23797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: Bootstrap complete.
I20260806 02:45:51.578241 23797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:51.579617 23797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 894be089543b4aa29b3edfcc5f3085e9, State: Initialized, Role: FOLLOWER
I20260806 02:45:51.580294 23797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 110, Last appended: 7.110, Last appended by leader: 110, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:51.581225 23806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:51.581655 23806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:51.582499 23797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:45:51.597523 23824 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:51.597651 23824 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: loading cluster ID for follower catalog manager: success
I20260806 02:45:51.603381 23824 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: acquiring CA information for follower catalog manager: success
I20260806 02:45:51.595388 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:45:51.604131 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:45:51.608430 23824 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 02:45:51.649292 21950 heartbeater.cc:347] Connected to a master server at 127.20.60.60:42309
I20260806 02:45:51.652567 23756 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" instance_seqno: 1785984321670316) as {username='slave'} at 127.20.60.3:33297; Asking this server to re-register.
I20260806 02:45:51.653403 21950 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:51.655342 23756 ts_manager.cc:194] Registered new tserver with Master: 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
I20260806 02:45:51.677736 23837 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a225e002d607451a94b3686f10aeea9a)
I20260806 02:45:51.677943 23837 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:51.687098 23837 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
I20260806 02:45:51.696962 21761 heartbeater.cc:347] Connected to a master server at 127.20.60.60:42309
I20260806 02:45:51.699412 23756 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" instance_seqno: 1785984320818216) as {username='slave'} at 127.20.60.2:53475; Asking this server to re-register.
I20260806 02:45:51.700173 21761 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:51.702116 23756 ts_manager.cc:194] Registered new tserver with Master: 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655)
I20260806 02:45:51.707952 23767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" candidate_term: 10 candidate_status { last_received { term: 9 index: 116 } } ignore_live_leader: false dest_uuid: "894be089543b4aa29b3edfcc5f3085e9" is_pre_election: true
W20260806 02:45:51.708577 23365 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:45:51.708688 23767 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfdbf4ef60014ea98bbcf9b6a9033f22 in term 7.
I20260806 02:45:51.709620 23366 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9, bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: a225e002d607451a94b3686f10aeea9a
I20260806 02:45:51.709954 23837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260806 02:45:51.710084 23837 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a225e002d607451a94b3686f10aeea9a)
I20260806 02:45:51.710188 23837 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 9 FOLLOWER]: Advancing to term 10
I20260806 02:45:51.713277 23837 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:51.714216 23837 leader_election.cc:288] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 election: Requested vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), 894be089543b4aa29b3edfcc5f3085e9 (127.20.60.60:42309)
I20260806 02:45:51.714783 23767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" candidate_term: 10 candidate_status { last_received { term: 9 index: 116 } } ignore_live_leader: false dest_uuid: "894be089543b4aa29b3edfcc5f3085e9"
I20260806 02:45:51.714998 23767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 7 FOLLOWER]: Advancing to term 10
W20260806 02:45:51.715386 23365 leader_election.cc:334] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:45:51.721614 23767 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfdbf4ef60014ea98bbcf9b6a9033f22 in term 10.
I20260806 02:45:51.722092 23366 leader_election.cc:302] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9, bfdbf4ef60014ea98bbcf9b6a9033f22; no voters: a225e002d607451a94b3686f10aeea9a
I20260806 02:45:51.722455 23837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 10 FOLLOWER]: Leader election won for term 10
I20260806 02:45:51.723778 23837 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [term 10 LEADER]: Becoming Leader. State: Replica: bfdbf4ef60014ea98bbcf9b6a9033f22, State: Running, Role: LEADER
I20260806 02:45:51.724370 23837 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 116, Committed index: 116, Last appended: 9.116, Last appended by leader: 116, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:51.726393 23845 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfdbf4ef60014ea98bbcf9b6a9033f22. Latest consensus state: current_term: 10 leader_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:51.726810 23845 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [sys.catalog]: This master's current role is: LEADER
I20260806 02:45:51.727530 23847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:45:51.731071 23847 catalog_manager.cc:669] Loaded metadata for table default.table_e41f56f2e4a1439ca6b8f2bca460a670 [id=1bbd1184e7754c5bb9ffdf82791c73fb]
I20260806 02:45:51.731756 23847 catalog_manager.cc:669] Loaded metadata for table default.table_97ee2606f8aa47dcb2c67a9ae29dd3fb [id=370d5895c35f4c96878270e71cac2443]
I20260806 02:45:51.732128 23847 catalog_manager.cc:669] Loaded metadata for table default.table_ea1a38afd92a4b15bfabb80e58f8c644 [id=383ba93bcba140c1b1bd2b36f8720b86]
I20260806 02:45:51.732533 23847 catalog_manager.cc:669] Loaded metadata for table default.table_d7249617a95049389b90db3bd551ace4 [id=4179f72ea45043e39ddce8e93c3429e9]
I20260806 02:45:51.732873 23847 catalog_manager.cc:669] Loaded metadata for table default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e]
I20260806 02:45:51.733233 23847 catalog_manager.cc:669] Loaded metadata for table default.table_0febb4673dff4e20ae8124efba9627c8 [id=5cb271f99441401cad4fd2e40e943a6e]
I20260806 02:45:51.733628 23847 catalog_manager.cc:669] Loaded metadata for table default.table_21221d53bf9f4c46a5ffe671a1ffe424 [id=610932bdd3f94b4e886d586c28ff6249]
I20260806 02:45:51.733982 23847 catalog_manager.cc:669] Loaded metadata for table default.table_e8df6dde49954df7b57567fc0cd12872 [id=a0cffb03ae7641f8848b8d2aab0fca8b]
I20260806 02:45:51.734309 23847 catalog_manager.cc:669] Loaded metadata for table default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26]
I20260806 02:45:51.734705 23847 catalog_manager.cc:669] Loaded metadata for table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022]
I20260806 02:45:51.735114 23847 catalog_manager.cc:669] Loaded metadata for table default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c]
I20260806 02:45:51.735601 23847 catalog_manager.cc:669] Loaded metadata for table default.table_00c06d2b7d6d40658a4390f4ee3bbab5 [id=c6a05485b0a54967ba8904aa969edd3f]
I20260806 02:45:51.736289 23847 catalog_manager.cc:669] Loaded metadata for table default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704]
I20260806 02:45:51.736755 23847 catalog_manager.cc:669] Loaded metadata for table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e]
I20260806 02:45:51.737121 23847 catalog_manager.cc:669] Loaded metadata for table default.table_c5be13d5ad2c4a479d2cc9074d14038f [id=f6939f0d45ae4f358d206a8cf2fe5db0]
I20260806 02:45:51.739717 23847 tablet_loader.cc:96] loaded metadata for tablet 043839a5eca143d5882f40faaab39692 (table default.table_21221d53bf9f4c46a5ffe671a1ffe424 [id=610932bdd3f94b4e886d586c28ff6249])
I20260806 02:45:51.740303 23847 tablet_loader.cc:96] loaded metadata for tablet 10189388393d47ce900b44cbad3ff425 (table default.table_0e9bb7fa1ed44ca0af8546b2c255fa62 [id=f5342be18d4e43b480c14cf440e6c66e])
I20260806 02:45:51.740764 23847 tablet_loader.cc:96] loaded metadata for tablet 237439c720344cfd97bd04bf8368a375 (table default.table_ea1a38afd92a4b15bfabb80e58f8c644 [id=383ba93bcba140c1b1bd2b36f8720b86])
I20260806 02:45:51.741092 23847 tablet_loader.cc:96] loaded metadata for tablet 290cc76d8c174a1fa3645f3bfe754b52 (table default.table_f243c93fbf57419bb6c685c08775e6ba [id=ba73c7eac7c245e598f7595efbc1e42c])
I20260806 02:45:51.741537 23847 tablet_loader.cc:96] loaded metadata for tablet 293b93e6322842aaaa786570f2810548 (table default.table_c453f255605e4419859a21f098c7243e [id=ad046cf7c81b42e5afcde9d116b3fc26])
I20260806 02:45:51.741991 23847 tablet_loader.cc:96] loaded metadata for tablet 3196f6ee70d34d298b7db56d9759fab7 (table default.table_00c06d2b7d6d40658a4390f4ee3bbab5 [id=c6a05485b0a54967ba8904aa969edd3f])
I20260806 02:45:51.742451 23847 tablet_loader.cc:96] loaded metadata for tablet 3d44c58ba1f044b38f10da10cec410e3 (table default.table_e41f56f2e4a1439ca6b8f2bca460a670 [id=1bbd1184e7754c5bb9ffdf82791c73fb])
I20260806 02:45:51.742875 23847 tablet_loader.cc:96] loaded metadata for tablet 50c46742a5a44d54aba65eec2f36aa09 (table default.table_e24bd493305d46a4977ed544ce47cc83 [id=b8a5ee6530364be1ae88e64a44dcc022])
I20260806 02:45:51.743378 23847 tablet_loader.cc:96] loaded metadata for tablet 62b1e9f4216845739fc757c975eb8c02 (table default.table_97ee2606f8aa47dcb2c67a9ae29dd3fb [id=370d5895c35f4c96878270e71cac2443])
I20260806 02:45:51.743806 23847 tablet_loader.cc:96] loaded metadata for tablet 7839cfd0a7fa44d3b6fefdfde3d9442b (table default.table_5cea2d6ea74c4c2f9366162462afcd71 [id=ee80e9ed535b43aaa5bd1dd782332704])
I20260806 02:45:51.744489 23847 tablet_loader.cc:96] loaded metadata for tablet 8243a6730a6e4939998608a4e4b63a18 (table default.table_ddb47e155c91465f8e38147105f2a6bd [id=4c9f8d8c878a4b128b71e594b536267e])
I20260806 02:45:51.744907 23847 tablet_loader.cc:96] loaded metadata for tablet a6e1420c19fc44b3901f653f137ec6c1 (table default.table_e8df6dde49954df7b57567fc0cd12872 [id=a0cffb03ae7641f8848b8d2aab0fca8b])
I20260806 02:45:51.746167 23847 tablet_loader.cc:96] loaded metadata for tablet cbfd41ea3624457da227733bea68ffc5 (table default.table_0febb4673dff4e20ae8124efba9627c8 [id=5cb271f99441401cad4fd2e40e943a6e])
I20260806 02:45:51.746965 23847 tablet_loader.cc:96] loaded metadata for tablet f9fd42a920b24117a4d687fec13ad17b (table default.table_d7249617a95049389b90db3bd551ace4 [id=4179f72ea45043e39ddce8e93c3429e9])
I20260806 02:45:51.747277 23847 tablet_loader.cc:96] loaded metadata for tablet fdc39b7dea2c43cebc730cffacd38d10 (table default.table_c5be13d5ad2c4a479d2cc9074d14038f [id=f6939f0d45ae4f358d206a8cf2fe5db0])
I20260806 02:45:51.747818 23847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:45:51.749635 23847 catalog_manager.cc:1259] Loaded cluster ID: 62f3bc067d714e7b8227b3a21d565ed8
I20260806 02:45:51.749729 23847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:45:51.751893 23847 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:45:51.753576 23847 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22: Loaded TSK: 0
I20260806 02:45:51.754494 23847 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:45:51.755976 23847 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260806 02:45:51.757606 23847 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: 29
I20260806 02:45:51.830871 23381 master_service.cc:946] ReplaceTablet: received request to replace tablet f9fd42a920b24117a4d687fec13ad17b from {username='slave'} at 127.0.0.1:40666
I20260806 02:45:51.831640 23377 master_service.cc:946] ReplaceTablet: received request to replace tablet 043839a5eca143d5882f40faaab39692 from {username='slave'} at 127.0.0.1:40666
I20260806 02:45:51.833771 23376 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40666:
table { table_name: "default.Table_0febb4673dff4e20ae8124efba9627c8" } modify_external_catalogs: true
I20260806 02:45:51.834254 23378 master_service.cc:946] ReplaceTablet: received request to replace tablet 043839a5eca143d5882f40faaab39692 from {username='slave'} at 127.0.0.1:40666
I20260806 02:45:51.834219 23376 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40666:
table { table_name: "default.Table_0febb4673dff4e20ae8124efba9627c8" } modify_external_catalogs: true
I20260806 02:45:51.834733 23380 master_service.cc:946] ReplaceTablet: received request to replace tablet 10189388393d47ce900b44cbad3ff425 from {username='slave'} at 127.0.0.1:40666
I20260806 02:45:51.834546 23372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40666:
name: "default.Table_cac77bd182314ee2a2068374ddeca8d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:51.834630 23379 master_service.cc:946] ReplaceTablet: received request to replace tablet f9fd42a920b24117a4d687fec13ad17b from {username='slave'} at 127.0.0.1:40666
I20260806 02:45:51.835546 23373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40666:
name: "default.Table_1ff877d50df44dccb8be0241b847e8c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:51.836302 23374 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40666:
name: "default.Table_468b3a35d3fb4572bc7b89d436d4e319"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:45:51.837007 23375 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40666:
name: "default.Table_3945346dd2514eddba4dfc965eb5d995"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260806 02:45:51.844903 23455 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 31): 31 DROP_TABLE default.table_1ff877d50df44dccb8be0241b847e8c9
*** Check failure stack trace: ***
@ 0x7fc6d23029ad google::LogMessage::Fail() at ??:0
@ 0x7fc6d230588a google::LogMessage::SendToLog() at ??:0
@ 0x7fc6d23024de google::LogMessage::Flush() at ??:0
@ 0x7fc6d23026d1 google::LogMessage::~LogMessage() at ??:0
I20260806 02:45:51.860855 23767 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Refusing update from remote peer bfdbf4ef60014ea98bbcf9b6a9033f22: Log matching property violated. Preceding OpId in replica: term: 7 index: 110. Preceding OpId from leader: term: 10 index: 118. (index mismatch)
@ 0x7fc6e7ea0ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260806 02:45:51.862098 23845 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 117, Last known committed idx: 110, Time since last communication: 0.000s
W20260806 02:45:51.871260 23365 consensus_peers.cc:487] T 00000000000000000000000000000000 P bfdbf4ef60014ea98bbcf9b6a9033f22 -> Peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Couldn't send request to peer a225e002d607451a94b3686f10aeea9a. Status: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7fc6e7e9f901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7fc6e7ea666c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7fc6e7ea64e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc6d4f2ceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc6d82066db start_thread at ??:0
@ 0x7fc6d119c61f clone at ??:0
I20260806 02:45:51.921176 23861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 10 leader_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:51.921689 23861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:51.921758 23851 mvcc.cc:205] Tried to move back new op lower bound from 7315391891013521408 to 7315391863647399936. Current Snapshot: MvccSnapshot[applied={T|T < 7315391891013521408}]
I20260806 02:45:51.922155 23806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfdbf4ef60014ea98bbcf9b6a9033f22. Latest consensus state: current_term: 10 leader_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:51.922482 23806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:51.923333 23862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 10 leader_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } } }
I20260806 02:45:51.923812 23862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:45:52.042701 23756 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" instance_seqno: 1785984319928949) as {username='slave'} at 127.20.60.1:47113; Asking this server to re-register.
I20260806 02:45:52.043653 21581 heartbeater.cc:464] Registering TS with master...
I20260806 02:45:52.045533 23756 ts_manager.cc:194] Registered new tserver with Master: 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:45:52.103931 21434 connection.cc:535] client connection to 127.20.60.61:36797 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:45:52.103873 23732 connection.cc:535] server connection from 127.0.0.1:49592 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:45:52.105176 21976 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.20.60.61:36797)
W20260806 02:45:52.105702 21976 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.20.60.61:36797)
W20260806 02:45:52.106120 21976 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.20.60.61:36797)
W20260806 02:45:52.106900 21976 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.20.60.61:36797)
I20260806 02:45:52.365691 23862 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:45:52.365875 23862 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:52.367511 23862 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:45:52.368595 23731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:45:52.370285 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:45:52.370620 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:45:52.370797 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:52.371152 23862 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:45:53.039856 23881 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:45:53.040076 23881 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:53.040853 23881 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:45:53.042794 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:45:53.042999 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:45:53.043157 23730 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:53.043414 23881 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:45:53.336181 23697 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:53.336385 23697 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:53.337281 23697 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:45:53.338263 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:45:53.338518 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:45:53.338665 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:53.338991 23697 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:45:53.667136 20720 master-stress-test.cc:578] Killing test thread 1/25
I20260806 02:45:53.747862 23893 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:45:53.748055 23893 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:53.748777 23893 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:45:53.751365 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:45:53.751602 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:45:53.751776 23730 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:53.752043 23893 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:45:53.752948 23890 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:45:53.753175 23890 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:53.753854 23890 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:45:53.754914 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:45:53.755213 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:45:53.755355 21436 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:45:53.755604 23890 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:45:53.967483 23890 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:45:53.967728 23890 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:53.968498 23890 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:45:53.969573 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:45:53.969796 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:45:53.969945 21434 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:45:53.970239 23890 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:45:54.808152 23911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:45:54.808442 23911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:54.810207 23911 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:45:54.811235 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:45:54.811399 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:45:54.811555 23730 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:54.811833 23911 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:45:56.605760 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:45:56.605957 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:45:56.823769 23945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:45:56.824066 23945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:45:56.824790 23945 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:45:56.826987 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:45:56.827220 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:45:56.827358 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:45:56.827683 23945 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:45:57.619542 23908 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:57.619768 23908 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:57.620397 23908 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:45:57.621380 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:45:57.621542 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:45:57.621655 21806 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:57.621881 23908 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:45:58.290738 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 6: this message will repeat every 5th retry.
I20260806 02:45:58.718865 23956 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:45:58.719095 23956 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:58.719807 23956 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:45:58.720805 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:45:58.721015 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:45:58.721200 21436 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:45:58.721500 23956 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:45:58.922775 23976 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:45:58.923048 23976 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:45:58.923833 23976 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:45:58.924762 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:45:58.924974 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:45:58.925150 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:45:58.925488 23976 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260806 02:45:59.835995 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 6: this message will repeat every 5th retry.
I20260806 02:45:59.844830 23908 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:45:59.845026 23908 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:45:59.846424 23908 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:45:59.846513 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:45:59.847185 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:45:59.847301 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:45:59.847617 23908 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:46:00.462466 23993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:46:00.462733 23993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:46:00.463444 23993 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:46:00.464531 23731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260806 02:46:00.465896 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:46:00.466132 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:46:00.466223 23730 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:46:00.466452 23993 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:46:01.607731 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:46:01.607963 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:46:02.685388 23908 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:46:02.685647 23908 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:46:02.686267 23908 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:46:02.687338 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:46:02.687526 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:46:02.687669 21806 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:46:02.687906 23908 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:46:02.953975 24028 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:46:02.954209 24028 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:46:02.954967 24028 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:46:02.956120 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:46:02.956336 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:46:02.956501 21434 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:46:02.956780 24028 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:46:03.596311 24041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:46:03.596508 24041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:46:03.597309 24041 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:46:03.599992 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:46:03.600239 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:46:03.600368 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:46:03.600629 24041 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:46:06.609751 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:46:06.609944 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:46:06.846582 24097 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:46:06.846778 24097 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:46:06.847644 24097 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:46:06.848543 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:46:06.848748 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:46:06.848855 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:46:06.849141 24097 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:46:06.936918 24097 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:46:06.937186 24097 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:46:06.938036 24097 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:46:06.939074 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:46:06.939280 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:46:06.939430 21434 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:46:06.939745 24097 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260806 02:46:08.257264 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 11: this message will repeat every 5th retry.
I20260806 02:46:08.391894 24112 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:46:08.392108 24112 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:46:08.392985 24112 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:46:08.394042 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:46:08.394292 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:46:08.394440 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:46:08.394750 24112 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:46:08.881721 24127 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:46:08.881930 24127 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:46:08.882673 24127 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:46:08.883865 23730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260806 02:46:08.885141 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:46:08.885324 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:46:08.885504 23730 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:46:08.885774 24127 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260806 02:46:09.955467 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 11: this message will repeat every 5th retry.
I20260806 02:46:11.611946 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:46:11.612171 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:46:12.021260 24168 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:46:12.021502 24168 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:46:12.022353 24168 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:46:12.024644 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:46:12.024884 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:46:12.024996 23730 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:46:12.025233 24168 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:46:15.555624 24221 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:46:15.555879 24221 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:46:15.556689 24221 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:46:15.557842 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:46:15.558048 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:46:15.558238 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:46:15.558575 24221 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:46:16.293464 24200 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:46:16.293677 24200 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:46:16.294407 24200 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:46:16.295378 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:46:16.295596 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:46:16.295699 21803 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:46:16.295935 24200 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:46:16.613852 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:46:16.614038 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:46:16.665190 24221 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:46:16.665493 24221 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:46:16.666272 24221 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:46:16.667491 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:46:16.667708 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:46:16.667862 21434 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:46:16.668187 24221 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260806 02:46:18.116369 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 16: this message will repeat every 5th retry.
W20260806 02:46:19.513166 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 16: this message will repeat every 5th retry.
I20260806 02:46:19.597666 24262 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:46:19.597931 24262 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:46:19.598716 24262 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:46:19.599782 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:46:19.600031 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:46:19.600214 21806 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:46:19.600499 24262 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:46:20.964253 24262 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:46:20.964542 24262 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:46:20.965363 24262 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:46:20.966432 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:46:20.966605 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:46:20.966792 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:46:20.967074 24262 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:46:21.615636 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:46:21.615900 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:46:22.230764 21583 heartbeater.cc:644] Failed to heartbeat to 127.20.60.61:36797 (92 consecutive failures): Network error: Failed to ping master at 127.20.60.61:36797: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:46:22.249495 21763 heartbeater.cc:644] Failed to heartbeat to 127.20.60.61:36797 (60 consecutive failures): Network error: Failed to ping master at 127.20.60.61:36797: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:46:22.749079 21951 heartbeater.cc:644] Failed to heartbeat to 127.20.60.62:33227 (71 consecutive failures): Network error: Failed to ping master at 127.20.60.62:33227: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:46:24.067353 24344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:46:24.067634 24344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:46:24.068492 24344 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:46:24.069705 23730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260806 02:46:24.071197 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:46:24.071434 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:46:24.071565 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:46:24.071866 24344 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:46:26.618649 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:46:26.618875 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:46:27.023378 24370 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:46:27.023627 24370 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:46:27.024469 24370 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:46:27.025651 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:46:27.025864 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:46:27.025969 21434 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:46:27.026278 24370 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:46:27.720312 24262 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:46:27.720513 24262 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:46:27.721293 24262 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:46:27.722186 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:46:27.722383 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:46:27.722570 21806 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:46:27.722832 24262 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:46:28.306454 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 21: this message will repeat every 5th retry.
I20260806 02:46:29.317310 24398 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:46:29.317592 24398 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:46:29.318364 24398 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:46:29.319336 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:46:29.319527 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:46:29.319711 21436 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:46:29.319993 24398 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260806 02:46:29.766561 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 21: this message will repeat every 5th retry.
I20260806 02:46:30.335501 24424 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:46:30.335742 24424 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:46:30.336370 24424 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:46:30.337601 23731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260806 02:46:30.339131 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:46:30.339395 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:46:30.339563 23730 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:46:30.339851 24424 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:46:31.620795 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:46:31.621050 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:46:33.688683 24467 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:46:33.688970 24467 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:46:33.689908 24467 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:46:33.690864 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:46:33.691082 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:46:33.691226 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:46:33.691550 24467 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:46:36.186988 24262 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:46:36.187186 24262 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:46:36.187948 24262 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:46:36.189077 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:46:36.189395 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:46:36.189594 21806 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:46:36.189899 24262 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:46:36.386171 24496 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:46:36.386415 24496 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:46:36.387259 24496 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:46:36.388298 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:46:36.388517 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:46:36.388669 21436 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:46:36.388967 24496 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:46:36.622764 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:46:36.622975 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:46:37.789390 24525 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:46:37.789654 24525 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:46:37.790567 24525 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:46:37.791648 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:46:37.791833 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:46:37.792037 21436 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:46:37.792315 24525 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260806 02:46:38.739895 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 26: this message will repeat every 5th retry.
I20260806 02:46:38.778322 24262 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:46:38.778577 24262 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:46:38.779340 24262 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:46:38.780510 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:46:38.780718 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:46:38.780902 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:46:38.781180 24262 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:46:39.671720 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 26: this message will repeat every 5th retry.
I20260806 02:46:41.129379 24262 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:46:41.129755 24262 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:46:41.130553 24262 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:46:41.131645 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:46:41.131876 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:46:41.132047 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:46:41.132368 24262 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:46:41.624915 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:46:41.625139 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:46:44.456583 24643 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:46:44.456846 24643 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:46:44.458288 24643 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:46:44.459156 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:46:44.459720 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:46:44.459885 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:46:44.460227 24643 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:46:46.627096 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:46:46.627482 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:46:46.725528 24698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:46:46.725848 24698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:46:46.727121 24698 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:46:46.727860 23731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260806 02:46:46.729873 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:46:46.730224 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:46:46.730472 23730 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:46:46.730829 24698 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260806 02:46:48.725869 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 31: this message will repeat every 5th retry.
I20260806 02:46:49.285612 24737 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:46:49.285831 24737 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:46:49.286528 24737 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:46:49.287505 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:46:49.287807 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:46:49.288013 21436 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:46:49.288257 24737 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260806 02:46:49.985281 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 31: this message will repeat every 5th retry.
I20260806 02:46:51.629555 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:46:51.629735 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:46:52.057802 24792 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:46:52.058043 24792 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:46:52.058866 24792 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:46:52.059986 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:46:52.060179 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:46:52.060544 21436 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:46:52.060814 24792 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:46:54.189571 24765 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:46:54.189888 24765 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:46:54.190871 24765 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:46:54.192124 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:46:54.192449 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:46:54.192607 21806 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:46:54.192902 24765 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:46:55.367337 24765 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:46:55.367666 24765 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:46:55.368613 24765 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:46:55.369764 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:46:55.370283 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:46:55.370538 21803 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:46:55.371006 24765 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:46:56.125385 24865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:46:56.125914 24865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:46:56.127445 24865 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:46:56.130441 23730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:46:56.130460 23731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260806 02:46:56.133554 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:46:56.133822 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:46:56.134156 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:46:56.134527 24865 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:46:56.304139 24864 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:46:56.304472 24864 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:46:56.306026 24864 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:46:56.307523 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:46:56.307868 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:46:56.308171 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:46:56.308634 24864 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:46:56.631004 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:46:56.631193 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:46:57.338802 24765 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:46:57.339030 24765 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:46:57.339708 24765 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:46:57.341110 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:46:57.341499 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:46:57.341668 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:46:57.342000 24765 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:46:58.201208 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 36: this message will repeat every 5th retry.
W20260806 02:46:59.417204 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 36: this message will repeat every 5th retry.
I20260806 02:47:00.126164 24938 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:47:00.126600 24938 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:47:00.127812 24938 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:47:00.128911 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:47:00.129158 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:47:00.129423 21434 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:47:00.129706 24938 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:47:00.318629 24939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:47:00.318946 24939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:47:00.319826 24939 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:47:00.322840 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:47:00.323161 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:47:00.323302 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:47:00.323596 24939 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:47:01.632655 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:47:01.632858 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:47:06.634658 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:47:06.634907 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:47:08.327152 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 41: this message will repeat every 5th retry.
I20260806 02:47:08.650830 25089 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:47:08.651158 25089 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:47:08.652119 25089 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:47:08.653090 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:47:08.653316 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:47:08.653625 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:47:08.653924 25089 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:47:09.178382 24765 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:47:09.178654 24765 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:47:09.179445 24765 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:47:09.180490 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:47:09.180761 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:47:09.180908 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:47:09.181273 24765 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:47:09.377063 25104 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:47:09.377300 25104 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:47:09.378207 25104 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:47:09.379688 23731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111) [suppressed 2 similar messages]
W20260806 02:47:09.381665 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:47:09.381866 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:47:09.382000 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:47:09.382395 25104 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260806 02:47:10.056672 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 41: this message will repeat every 5th retry.
I20260806 02:47:11.637074 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:47:11.637375 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:47:12.563287 24765 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:47:12.563562 24765 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:47:12.564603 24765 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:47:12.566064 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:47:12.566260 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:47:12.566416 21803 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:47:12.566705 24765 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:47:12.762989 24765 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:47:12.763283 24765 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:47:12.764091 24765 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:47:12.765213 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:47:12.765559 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:47:12.765799 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:47:12.766206 24765 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:47:13.171656 25160 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:47:13.172163 25160 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:47:13.173096 25160 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:47:13.174401 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:47:13.174692 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:47:13.174854 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:47:13.175149 25160 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:47:16.639007 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:47:16.639245 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:47:16.765614 25232 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:47:16.765991 25232 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:47:16.767136 25232 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:47:16.768242 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:47:16.768442 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:47:16.768659 21436 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:47:16.768939 25232 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260806 02:47:17.736994 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 46: this message will repeat every 5th retry.
I20260806 02:47:19.157372 25192 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:47:19.157691 25192 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:47:19.158509 25192 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:47:19.159567 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:47:19.159780 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:47:19.159886 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:47:19.160128 25192 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:47:20.861303 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 46: this message will repeat every 5th retry.
I20260806 02:47:21.640946 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:47:21.641177 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:47:22.234306 25330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:47:22.234547 25330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:47:22.235371 25330 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:47:22.236722 23731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260806 02:47:22.238282 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:47:22.238523 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:47:22.238672 23730 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:47:22.239032 25330 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260806 02:47:22.313017 21583 heartbeater.cc:644] Failed to heartbeat to 127.20.60.61:36797 (277 consecutive failures): Network error: Failed to ping master at 127.20.60.61:36797: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:47:22.471652 21762 heartbeater.cc:644] Failed to heartbeat to 127.20.60.62:33227 (190 consecutive failures): Network error: Failed to ping master at 127.20.60.62:33227: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:47:22.975559 21951 heartbeater.cc:644] Failed to heartbeat to 127.20.60.62:33227 (191 consecutive failures): Network error: Failed to ping master at 127.20.60.62:33227: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:47:24.772413 25363 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:47:24.772745 25363 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:47:24.773609 25363 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:47:24.774681 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:47:24.774879 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:47:24.775053 21436 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:47:24.775352 25363 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:47:25.092990 25192 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:47:25.093222 25192 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:47:25.094101 25192 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:47:25.095141 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:47:25.095357 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:47:25.095475 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:47:25.095767 25192 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:47:26.642935 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:47:26.643178 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:47:26.729215 25400 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:47:26.729450 25400 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:47:26.730186 25400 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:47:26.731370 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:47:26.731639 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:47:26.731771 21436 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:47:26.732051 25400 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260806 02:47:27.129462 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 51: this message will repeat every 5th retry.
I20260806 02:47:28.143788 25436 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:47:28.144037 25436 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:47:28.144817 25436 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:47:28.146061 23730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260806 02:47:28.147909 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:47:28.148165 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:47:28.148306 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:47:28.148612 25436 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:47:28.793311 25446 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:47:28.793612 25446 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:47:28.794447 25446 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:47:28.798134 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:47:28.798395 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:47:28.798556 23730 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:47:28.798892 25446 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:47:29.172514 25192 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:47:29.172824 25192 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:47:29.173816 25192 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:47:29.174880 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:47:29.175176 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:47:29.175319 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:47:29.175639 25192 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:47:29.804908 25433 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:47:29.805187 25433 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:47:29.806025 25433 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:47:29.807163 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:47:29.807350 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:47:29.807659 21436 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:47:29.807991 25433 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:47:30.508911 25192 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:47:30.509109 25192 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:47:30.509819 25192 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:47:30.510869 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:47:30.511075 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:47:30.511168 21803 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:47:30.511407 25192 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:47:30.713290 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 51: this message will repeat every 5th retry.
I20260806 02:47:30.745290 25192 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:47:30.745533 25192 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:47:30.746366 25192 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:47:30.747543 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:47:30.747798 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:47:30.747990 21806 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:47:30.748240 25192 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:47:31.645114 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:47:31.645411 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:47:35.341053 25192 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:47:35.341387 25192 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:47:35.342084 25192 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:47:35.343339 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:47:35.343593 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:47:35.343757 21803 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:47:35.344066 25192 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:47:36.647060 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:47:36.647249 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:47:37.016934 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 56: this message will repeat every 5th retry.
I20260806 02:47:37.758522 25595 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:47:37.758811 25595 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:47:37.760165 25595 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:47:37.761286 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:47:37.761570 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:47:37.761735 21434 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:47:37.762089 25595 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:47:39.998796 25625 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:47:39.999086 25625 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:47:40.000141 25625 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:47:40.001216 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:47:40.001410 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:47:40.001570 21436 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:47:40.001989 25625 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:47:40.314953 25192 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:47:40.315296 25192 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:47:40.316063 25192 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:47:40.316910 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:47:40.317103 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:47:40.317230 21803 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:47:40.317468 25192 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:47:41.274434 25192 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:47:41.274621 25192 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:47:41.275400 25192 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:47:41.276434 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:47:41.276697 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:47:41.276909 21806 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:47:41.277220 25192 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:47:41.648974 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:47:41.649199 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:47:41.739670 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 56: this message will repeat every 5th retry.
I20260806 02:47:43.122097 25680 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:47:43.122349 25680 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:47:43.123152 25680 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:47:43.124145 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:47:43.124325 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:47:43.124481 21434 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:47:43.124766 25680 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:47:43.652896 25192 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:47:43.653184 25192 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:47:43.654036 25192 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:47:43.655131 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:47:43.655377 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:47:43.655498 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:47:43.655788 25192 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:47:46.650952 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:47:46.651242 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:47:47.138809 25775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:47:47.139123 25775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:47:47.140264 25775 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:47:47.141017 23730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260806 02:47:47.142900 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:47:47.143188 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:47:47.143343 23730 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:47:47.143640 25775 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260806 02:47:47.878253 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 61: this message will repeat every 5th retry.
I20260806 02:47:51.509073 25847 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:47:51.509302 25847 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:47:51.510180 25847 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:47:51.511278 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:47:51.511584 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:47:51.511778 21434 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:47:51.512115 25847 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:47:51.624930 23737 maintenance_manager.cc:419] P 894be089543b4aa29b3edfcc5f3085e9: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033345
I20260806 02:47:51.670843 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:47:51.671093 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:47:51.682078 25847 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:47:51.682376 25847 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:47:51.683422 25847 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:47:51.684549 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:47:51.684718 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:47:51.684826 21436 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:47:51.685092 25847 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:47:51.703722 23735 maintenance_manager.cc:643] P 894be089543b4aa29b3edfcc5f3085e9: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.078s user 0.042s sys 0.008s Metrics: {"bytes_written":23897,"cfile_init":1,"dirs.queue_time_us":746,"dirs.run_cpu_time_us":243,"dirs.run_wall_time_us":1014,"drs_written":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1450,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":60,"thread_start_us":869,"threads_started":2,"wal-append.queue_time_us":483}
I20260806 02:47:51.706802 23737 maintenance_manager.cc:419] P 894be089543b4aa29b3edfcc5f3085e9: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4540 bytes on disk
I20260806 02:47:51.707748 23735 maintenance_manager.cc:643] P 894be089543b4aa29b3edfcc5f3085e9: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":80,"lbm_reads_lt_1ms":4}
W20260806 02:47:52.139525 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 61: this message will repeat every 5th retry.
I20260806 02:47:53.030210 25874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:47:53.030473 25874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:47:53.031363 25874 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:47:53.033023 23731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260806 02:47:53.035392 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:47:53.035624 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:47:53.035836 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:47:53.036141 25874 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:47:54.227173 25893 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:47:54.227402 25893 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:47:54.228395 25893 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:47:54.229553 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:47:54.229735 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:47:54.229872 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:47:54.230162 25893 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:47:54.267761 25893 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:47:54.268036 25893 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:47:54.268766 25893 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:47:54.269846 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:47:54.270517 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:47:54.270654 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:47:54.271004 25893 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:47:56.672742 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:47:56.672914 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:47:57.896178 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 66: this message will repeat every 5th retry.
I20260806 02:47:58.376967 25893 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:47:58.377175 25893 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:47:58.378062 25893 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:47:58.379235 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:47:58.379469 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:47:58.379632 21806 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:47:58.380002 25893 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:47:58.894819 25893 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:47:58.895092 25893 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:47:58.895795 25893 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:47:58.896948 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:47:58.897246 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:47:58.897449 21806 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:47:58.897753 25893 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:47:59.346244 25893 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:47:59.346562 25893 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:47:59.347355 25893 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:47:59.348431 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:47:59.348762 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:47:59.348928 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:47:59.349239 25893 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:48:01.674641 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:48:01.674893 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:48:01.986866 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 66: this message will repeat every 5th retry.
I20260806 02:48:04.100124 26091 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:48:04.100514 26091 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:48:04.101794 26091 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:48:04.102834 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:48:04.103044 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:48:04.103209 21434 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:48:04.103485 26091 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:48:06.676577 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:48:06.676868 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:48:07.024366 26155 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:48:07.024636 26155 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:48:07.025816 26155 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:48:07.027427 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:48:07.027702 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:48:07.027870 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:48:07.028167 26155 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260806 02:48:07.505633 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 71: this message will repeat every 5th retry.
I20260806 02:48:07.931814 25893 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:48:07.932160 25893 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:48:07.932963 25893 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:48:07.934018 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:48:07.934226 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:48:07.934331 21803 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:48:07.934635 25893 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:48:07.962569 26184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:48:07.962857 26184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:48:07.963647 26184 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:48:07.964887 23731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260806 02:48:07.966465 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:48:07.966950 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:48:07.967087 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:48:07.967453 26184 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:48:11.051573 25893 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:48:11.051761 25893 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:48:11.052434 25893 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:48:11.053422 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:48:11.053771 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:48:11.053987 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:48:11.054450 25893 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:48:11.508179 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 71: this message will repeat every 5th retry.
I20260806 02:48:11.678851 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:48:11.679065 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:48:13.038565 26283 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:48:13.038826 26283 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:48:13.039633 26283 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:48:13.040690 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:48:13.040956 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:48:13.041112 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:48:13.041461 26283 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:48:15.836805 26307 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:48:15.837096 26307 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:48:15.837898 26307 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:48:15.838920 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:48:15.839151 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:48:15.839314 21436 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:48:15.839615 26307 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:48:16.600455 25893 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:48:16.600790 25893 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:48:16.601805 25893 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:48:16.602834 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:48:16.603128 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:48:16.603444 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:48:16.603763 25893 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:48:16.681442 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:48:16.681663 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:48:18.069023 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 76: this message will repeat every 5th retry.
W20260806 02:48:21.090395 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 76: this message will repeat every 5th retry.
I20260806 02:48:21.683367 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:48:21.683632 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:48:21.869822 26402 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:48:21.870155 26402 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:48:21.871284 26402 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:48:21.872236 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:48:21.872473 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:48:21.872701 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:48:21.872998 26402 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:48:22.780067 21582 heartbeater.cc:644] Failed to heartbeat to 127.20.60.62:33227 (478 consecutive failures): Network error: Failed to ping master at 127.20.60.62:33227: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:48:22.787004 21763 heartbeater.cc:644] Failed to heartbeat to 127.20.60.61:36797 (300 consecutive failures): Network error: Failed to ping master at 127.20.60.61:36797: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:48:22.809026 26484 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:48:22.809259 26484 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:48:22.810364 26484 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:48:22.811483 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:48:22.811786 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:48:22.811923 21436 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:48:22.812199 26484 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260806 02:48:23.289767 21952 heartbeater.cc:644] Failed to heartbeat to 127.20.60.61:36797 (301 consecutive failures): Network error: Failed to ping master at 127.20.60.61:36797: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:48:23.900856 26402 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:48:23.901214 26402 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:48:23.902268 26402 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:48:23.903417 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:48:23.903633 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:48:23.903882 21806 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:48:23.904314 26524 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:48:24.472185 26532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:48:24.472582 26532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:48:24.473878 26532 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:48:24.475890 23730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260806 02:48:24.479070 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:48:24.479365 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:48:24.479569 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:48:24.479918 26532 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:48:25.507768 26524 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:48:25.508011 26524 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:48:25.508635 26524 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:48:25.509817 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:48:25.509987 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:48:25.510174 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:48:25.510525 26524 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:48:26.685691 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:48:26.685897 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:48:27.949849 26594 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:48:27.950064 26594 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:48:27.951069 26594 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:48:27.952512 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:48:27.952775 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:48:27.952986 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:48:27.953323 26594 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260806 02:48:28.262454 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 81: this message will repeat every 5th retry.
W20260806 02:48:31.391592 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 81: this message will repeat every 5th retry.
I20260806 02:48:31.688109 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:48:31.688360 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:48:35.172730 26771 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:48:35.173007 26771 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:48:35.174049 26771 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:48:35.175376 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:48:35.175678 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:48:35.175882 21436 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:48:35.176240 26771 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:48:36.068150 26783 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:48:36.068605 26783 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:48:36.069487 26783 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:48:36.070571 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:48:36.070763 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:48:36.070943 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:48:36.071266 26783 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:48:36.349552 26660 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:48:36.349782 26660 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:48:36.350750 26660 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:48:36.351743 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:48:36.351945 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:48:36.352140 21806 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:48:36.352415 26660 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:48:36.690178 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:48:36.690400 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:48:38.172456 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 86: this message will repeat every 5th retry.
I20260806 02:48:38.281158 26805 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:48:38.281514 26805 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:48:38.282541 26805 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:48:38.283885 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:48:38.284205 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:48:38.284346 21806 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:48:38.284833 26805 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:48:41.457047 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 86: this message will repeat every 5th retry.
I20260806 02:48:41.692158 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:48:41.692423 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:48:43.451277 26924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:48:43.451628 26924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:48:43.452621 26924 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:48:43.453866 23730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260806 02:48:43.455551 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:48:43.455910 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:48:43.456038 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:48:43.456422 26924 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:48:43.866993 26873 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:48:43.867372 26873 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:48:43.868194 26873 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:48:43.869175 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:48:43.869804 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:48:43.869989 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:48:43.870659 26873 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:48:46.634541 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 91: this message will repeat every 5th retry.
I20260806 02:48:46.694849 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:48:46.695097 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:48:47.033975 26873 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:48:47.034303 26873 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:48:47.035136 26873 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:48:47.036154 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:48:47.036352 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:48:47.036540 21803 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:48:47.036875 26873 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:48:48.318857 26985 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:48:48.319103 26985 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:48:48.320173 26985 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:48:48.321473 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:48:48.321692 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:48:48.321808 21436 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:48:48.322045 26985 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:48:49.982633 27034 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:48:49.982908 27034 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:48:49.983925 27034 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:48:49.984997 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:48:49.985181 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:48:49.985386 21436 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:48:49.985707 27034 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:48:50.555369 26873 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:48:50.555745 26873 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:48:50.556497 26873 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:48:50.557641 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:48:50.558033 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:48:50.558257 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:48:50.558658 26873 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:48:51.119215 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 91: this message will repeat every 5th retry.
I20260806 02:48:51.697299 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:48:51.697572 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:48:56.699167 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:48:56.699361 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:48:56.799813 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 96: this message will repeat every 5th retry.
I20260806 02:48:57.361718 26873 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:48:57.362012 26873 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:48:57.362959 26873 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:48:57.364239 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:48:57.364558 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:48:57.364781 21806 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:48:57.365036 26873 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:48:58.288539 26873 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:48:58.288830 26873 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:48:58.289731 26873 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:48:58.290858 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:48:58.291150 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:48:58.291323 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:48:58.291654 26873 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:49:01.314764 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 96: this message will repeat every 5th retry.
I20260806 02:49:01.319444 27289 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:49:01.319763 27289 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:49:01.321081 27289 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:49:01.322834 23730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260806 02:49:01.325214 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:49:01.325479 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:49:01.325623 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:49:01.325989 27289 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:49:01.487344 26873 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:49:01.487608 26873 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:49:01.488288 26873 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:49:01.489539 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:49:01.489749 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:49:01.489852 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:49:01.490083 26873 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:49:01.700851 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:49:01.701126 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:49:01.777060 27287 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:49:01.777273 27287 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:49:01.778151 27287 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:49:01.779259 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:49:01.779543 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:49:01.779671 21434 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:49:01.779995 27287 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:49:02.627671 27330 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:49:02.628017 27330 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:49:02.629132 27330 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:49:02.630450 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:49:02.630667 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:49:02.630843 21806 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:49:02.631163 27330 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:49:04.772903 27351 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:49:04.773128 27351 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:49:04.773978 27351 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:49:04.775051 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:49:04.775243 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:49:04.775375 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:49:04.775648 27351 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:49:05.334728 27391 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:49:05.334964 27391 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:49:05.335891 27391 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:49:05.336990 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:49:05.337323 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:49:05.337637 21436 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:49:05.337893 27391 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:49:06.702960 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:49:06.703224 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:49:07.417945 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 101: this message will repeat every 5th retry.
I20260806 02:49:07.492278 27440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:49:07.492484 27440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:49:07.493158 27440 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:49:07.494971 23731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260806 02:49:07.496428 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:49:07.496694 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:49:07.496901 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:49:07.497375 27440 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:49:09.683410 27486 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:49:09.683722 27486 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:49:09.684798 27486 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:49:09.686041 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:49:09.686244 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:49:09.686405 21436 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:49:09.686827 27486 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:49:11.705322 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:49:11.705674 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:49:11.903254 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 101: this message will repeat every 5th retry.
I20260806 02:49:13.335605 27351 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:49:13.335939 27351 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:49:13.336767 27351 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:49:13.337862 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:49:13.338100 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:49:13.338210 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:49:13.338502 27351 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:49:14.128630 27581 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:49:14.128834 27581 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:49:14.129591 27581 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:49:14.130414 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:49:14.130573 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:49:14.130738 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:49:14.131033 27581 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:49:15.024410 27568 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:49:15.024562 27568 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:49:15.025182 27568 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:49:15.026084 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:49:15.026255 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:49:15.026376 21803 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:49:15.026623 27568 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:49:16.438200 27622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:49:16.438410 27622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:49:16.439328 27622 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:49:16.440586 23731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260806 02:49:16.442154 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:49:16.442327 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:49:16.442464 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:49:16.442734 27622 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:49:16.707048 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:49:16.707265 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:49:17.237253 27645 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:49:17.237589 27645 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:49:17.238399 27645 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:49:17.240639 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:49:17.240833 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:49:17.240988 23730 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:49:17.241294 27645 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260806 02:49:17.988616 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 106: this message will repeat every 5th retry.
I20260806 02:49:18.481935 27657 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:49:18.482242 27657 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:49:18.482990 27657 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:49:18.484082 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:49:18.484375 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:49:18.484498 21434 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:49:18.484768 27657 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:49:20.312716 27568 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:49:20.313004 27568 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:49:20.313804 27568 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:49:20.314961 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:49:20.315165 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:49:20.315291 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:49:20.315585 27568 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:49:21.517395 27720 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:49:21.517643 27720 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:49:21.518602 27720 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:49:21.519719 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:49:21.519834 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:49:21.519990 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:49:21.520258 27720 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:49:21.709025 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:49:21.709364 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:49:22.039716 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 106: this message will repeat every 5th retry.
W20260806 02:49:22.925697 21582 heartbeater.cc:644] Failed to heartbeat to 127.20.60.62:33227 (663 consecutive failures): Network error: Failed to ping master at 127.20.60.62:33227: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:49:23.059764 21763 heartbeater.cc:644] Failed to heartbeat to 127.20.60.61:36797 (420 consecutive failures): Network error: Failed to ping master at 127.20.60.61:36797: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:49:23.562778 21951 heartbeater.cc:644] Failed to heartbeat to 127.20.60.62:33227 (431 consecutive failures): Network error: Failed to ping master at 127.20.60.62:33227: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:49:26.710990 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:49:26.711217 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:49:27.928176 27568 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:49:27.928452 27568 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:49:27.929185 27568 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:49:27.930267 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:49:27.930474 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:49:27.930651 21806 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:49:27.930950 27568 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:49:27.952232 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 111: this message will repeat every 5th retry.
I20260806 02:49:30.638468 27568 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:49:30.638706 27568 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:49:30.640062 27568 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:49:30.640964 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:49:30.641144 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:49:30.641384 21806 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:49:30.641654 27568 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:49:31.135712 27916 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:49:31.135939 27916 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:49:31.137120 27916 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:49:31.138180 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:49:31.138329 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:49:31.138427 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:49:31.138716 27916 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:49:31.713382 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:49:31.713711 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:49:32.698886 27943 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:49:32.699079 27943 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:49:32.699792 27943 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:49:32.701094 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:49:32.701282 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:49:32.701400 21803 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:49:32.701619 27943 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:49:33.108784 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 111: this message will repeat every 5th retry.
I20260806 02:49:34.914009 28004 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:49:34.914244 28004 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:49:34.914980 28004 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:49:34.916026 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:49:34.916157 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:49:34.916244 21434 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:49:34.916607 28004 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:49:36.715129 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:49:36.715327 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:49:36.997748 28044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:49:36.997958 28044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:49:36.998950 28044 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:49:36.999758 23730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260806 02:49:37.001654 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:49:37.001827 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:49:37.002004 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:49:37.002362 28044 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:49:37.480763 28041 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:49:37.480979 28041 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:49:37.482048 28041 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:49:37.483167 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:49:37.483321 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:49:37.483443 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:49:37.483702 28041 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260806 02:49:37.904055 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 116: this message will repeat every 5th retry.
I20260806 02:49:41.717211 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:49:41.717514 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:49:42.398164 28122 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:49:42.398413 28122 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:49:42.399192 28122 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:49:42.400257 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:49:42.400416 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:49:42.400555 21806 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:49:42.400811 28122 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:49:42.721830 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 116: this message will repeat every 5th retry.
I20260806 02:49:43.348892 28122 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:49:43.349099 28122 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:49:43.349812 28122 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:49:43.351017 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:49:43.351190 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:49:43.351285 21806 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:49:43.351490 28122 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:49:43.652796 28162 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:49:43.653036 28162 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:49:43.653708 28162 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:49:43.654914 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:49:43.655057 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:49:43.655149 21436 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:49:43.655365 28162 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:49:44.668335 28179 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:49:44.668581 28179 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:49:44.669392 28179 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:49:44.670248 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:49:44.670403 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:49:44.670533 21436 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:49:44.670827 28179 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:49:46.718820 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:49:46.719053 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:49:47.517879 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 121: this message will repeat every 5th retry.
I20260806 02:49:48.459175 28239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:49:48.459380 28239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:49:48.460211 28239 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:49:48.461413 23731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260806 02:49:48.462677 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:49:48.462914 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:49:48.463138 23730 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:49:48.463462 28239 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:49:49.453276 28207 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:49:49.453553 28207 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:49:49.454469 28207 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:49:49.455473 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:49:49.455631 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:49:49.455788 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:49:49.456085 28207 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:49:49.701771 28207 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:49:49.701983 28207 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:49:49.702610 28207 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:49:49.703512 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:49:49.703708 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:49:49.703848 21803 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:49:49.704056 28207 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:49:51.720870 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:49:51.721179 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:49:51.973114 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 121: this message will repeat every 5th retry.
I20260806 02:49:52.339069 28297 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:49:52.339324 28297 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:49:52.340051 28297 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:49:52.341466 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:49:52.341670 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:49:52.341768 21436 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:49:52.341966 28297 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:49:53.698601 28328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:49:53.699139 28328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:49:53.700166 28328 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:49:53.701556 23731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260806 02:49:53.707497 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:49:53.707754 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:49:53.707895 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:49:53.708300 28328 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:49:54.049625 28340 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:49:54.049834 28340 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:49:54.050830 28340 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:49:54.051857 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:49:54.051985 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:49:54.052127 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:49:54.052372 28340 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:49:56.722675 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:49:56.722890 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:49:57.694490 28362 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:49:57.694892 28362 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:49:57.695820 28362 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:49:57.696854 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:49:57.696983 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:49:57.697129 21803 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:49:57.697475 28362 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:49:58.436991 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 126: this message will repeat every 5th retry.
I20260806 02:49:59.094702 28362 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:49:59.094901 28362 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:49:59.095743 28362 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:49:59.096714 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:49:59.096829 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:49:59.096977 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:49:59.097211 28362 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:50:00.900941 28452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:50:00.901165 28452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:50:00.901870 28452 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:50:00.903316 23731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260806 02:50:00.905231 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:50:00.905567 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
I20260806 02:50:00.905803 23730 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:50:00.906178 28452 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:50:01.724448 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:50:01.724710 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:50:02.145196 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 126: this message will repeat every 5th retry.
I20260806 02:50:02.491369 28482 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:50:02.491644 28482 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:50:02.492511 28482 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:50:02.493892 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:50:02.494048 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:50:02.494143 21436 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:50:02.494426 28482 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:50:02.759445 28362 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:50:02.759670 28362 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:50:02.760435 28362 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:50:02.761489 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:50:02.761648 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:50:02.761790 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:50:02.762076 28362 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:50:03.454639 28500 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:50:03.454804 28500 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:50:03.455410 28500 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:50:03.456265 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:50:03.456393 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:50:03.456560 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:50:03.456800 28500 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:50:06.711752 28562 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:50:06.711942 28562 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:50:06.712677 28562 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:50:06.713994 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:50:06.714186 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:50:06.714322 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:50:06.714581 28562 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:50:06.726295 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:50:06.726455 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:50:09.071815 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 131: this message will repeat every 5th retry.
I20260806 02:50:10.610638 28670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:50:10.610896 28670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:50:10.611774 28670 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:50:10.613510 23731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260806 02:50:10.615675 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:50:10.615863 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:50:10.616052 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:50:10.616350 28670 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:50:11.727985 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:50:11.728223 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 02:50:11.842715 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 131: this message will repeat every 5th retry.
I20260806 02:50:15.948596 28677 raft_consensus.cc:482] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:50:15.948830 28677 raft_consensus.cc:504] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:50:15.949551 28677 leader_election.cc:288] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:50:15.950626 21803 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
W20260806 02:50:15.950760 21806 leader_election.cc:341] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 290cc76d8c174a1fa3645f3bfe754b52
I20260806 02:50:15.950908 21806 leader_election.cc:302] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:50:15.951170 28677 raft_consensus.cc:2726] T 290cc76d8c174a1fa3645f3bfe754b52 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:50:16.729985 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:50:16.730187 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:50:17.256431 28782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:50:17.256618 28782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:50:17.257328 28782 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:50:17.258584 23730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260806 02:50:17.259939 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:50:17.260154 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:50:17.260331 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:50:17.260661 28782 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260806 02:50:19.040786 28826 raft_consensus.cc:482] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:50:19.041035 28826 raft_consensus.cc:504] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:50:19.041810 28826 leader_election.cc:288] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:50:19.042810 21806 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
W20260806 02:50:19.042946 21803 leader_election.cc:341] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: cbfd41ea3624457da227733bea68ffc5
I20260806 02:50:19.043118 21803 leader_election.cc:302] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:50:19.043470 28826 raft_consensus.cc:2726] T cbfd41ea3624457da227733bea68ffc5 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:50:19.155103 28829 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader bfdbf4ef60014ea98bbcf9b6a9033f22)
I20260806 02:50:19.155493 28829 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a225e002d607451a94b3686f10aeea9a" member_type: VOTER last_known_addr { host: "127.20.60.62" port: 33227 } } peers { permanent_uuid: "bfdbf4ef60014ea98bbcf9b6a9033f22" member_type: VOTER last_known_addr { host: "127.20.60.61" port: 36797 } } peers { permanent_uuid: "894be089543b4aa29b3edfcc5f3085e9" member_type: VOTER last_known_addr { host: "127.20.60.60" port: 42309 } }
I20260806 02:50:19.156191 28829 leader_election.cc:288] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227), bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797)
W20260806 02:50:19.158524 23730 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer bfdbf4ef60014ea98bbcf9b6a9033f22 (127.20.60.61:36797): Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:50:19.158687 23731 leader_election.cc:334] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a225e002d607451a94b3686f10aeea9a (127.20.60.62:33227): Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:50:19.158823 23731 leader_election.cc:302] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 894be089543b4aa29b3edfcc5f3085e9; no voters: a225e002d607451a94b3686f10aeea9a, bfdbf4ef60014ea98bbcf9b6a9033f22
I20260806 02:50:19.159052 28829 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 894be089543b4aa29b3edfcc5f3085e9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260806 02:50:19.685402 21436 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Couldn't send request to peer 88dc3ab284a64961b1fbdb5c53d1267f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 136: this message will repeat every 5th retry.
I20260806 02:50:20.243723 28844 raft_consensus.cc:482] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:50:20.243939 28844 raft_consensus.cc:504] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:50:20.244784 28844 leader_election.cc:288] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:50:20.245853 21434 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
W20260806 02:50:20.246032 21436 leader_election.cc:341] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 6cc50b1bfe414464b1863c4e26f295fb
I20260806 02:50:20.246142 21436 leader_election.cc:302] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:50:20.246419 28844 raft_consensus.cc:2726] T 6cc50b1bfe414464b1863c4e26f295fb P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:50:21.731765 23701 master_runner.cc:186] Error getting master registration for 127.20.60.62:33227: Network error: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111),
I20260806 02:50:21.731949 23701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:50:21.880494 28867 raft_consensus.cc:482] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88dc3ab284a64961b1fbdb5c53d1267f)
I20260806 02:50:21.880718 28867 raft_consensus.cc:504] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } }
I20260806 02:50:21.881425 28867 leader_election.cc:288] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883)
W20260806 02:50:21.882329 21436 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
W20260806 02:50:21.882508 21434 leader_election.cc:341] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Not found: Tablet not found: ba32fbee948d4adf96164d36673a1db6
I20260806 02:50:21.882663 21434 leader_election.cc:302] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48b0c1e2106b4d9b941929ddc6d97dd5; no voters: 88dc3ab284a64961b1fbdb5c53d1267f, 8ebd96b903b944f3b06851324b737bd5
I20260806 02:50:21.882958 28867 raft_consensus.cc:2726] T ba32fbee948d4adf96164d36673a1db6 P 48b0c1e2106b4d9b941929ddc6d97dd5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260806 02:50:21.979063 21434 consensus_peers.cc:487] T 6061f59aedec4e0a9b96239ebe6fa0df P 48b0c1e2106b4d9b941929ddc6d97dd5 -> Peer 8ebd96b903b944f3b06851324b737bd5 (127.20.60.3:43883): Couldn't send request to peer 8ebd96b903b944f3b06851324b737bd5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6061f59aedec4e0a9b96239ebe6fa0df. This is attempt 136: this message will repeat every 5th retry.
W20260806 02:50:23.029309 21582 heartbeater.cc:644] Failed to heartbeat to 127.20.60.62:33227 (844 consecutive failures): Network error: Failed to ping master at 127.20.60.62:33227: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
W20260806 02:50:23.317582 21763 heartbeater.cc:644] Failed to heartbeat to 127.20.60.61:36797 (540 consecutive failures): Network error: Failed to ping master at 127.20.60.61:36797: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:50:23.823635 21951 heartbeater.cc:644] Failed to heartbeat to 127.20.60.62:33227 (551 consecutive failures): Network error: Failed to ping master at 127.20.60.62:33227: Client connection negotiation failed: client connection to 127.20.60.62:33227: connect: Connection refused (error 111)
I20260806 02:50:24.784624 28884 raft_consensus.cc:482] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:50:24.784817 28884 raft_consensus.cc:504] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ebd96b903b944f3b06851324b737bd5" member_type: VOTER last_known_addr { host: "127.20.60.3" port: 43883 } } peers { permanent_uuid: "88dc3ab284a64961b1fbdb5c53d1267f" member_type: VOTER last_known_addr { host: "127.20.60.2" port: 40655 } } peers { permanent_uuid: "48b0c1e2106b4d9b941929ddc6d97dd5" member_type: VOTER last_known_addr { host: "127.20.60.1" port: 36593 } }
I20260806 02:50:24.785852 28884 leader_election.cc:288] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655), 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593)
W20260806 02:50:24.786957 21806 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88dc3ab284a64961b1fbdb5c53d1267f (127.20.60.2:40655): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
W20260806 02:50:24.787103 21803 leader_election.cc:341] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48b0c1e2106b4d9b941929ddc6d97dd5 (127.20.60.1:36593): Not found: Tablet not found: 8243a6730a6e4939998608a4e4b63a18
I20260806 02:50:24.787194 21803 leader_election.cc:302] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ebd96b903b944f3b06851324b737bd5; no voters: 48b0c1e2106b4d9b941929ddc6d97dd5, 88dc3ab284a64961b1fbdb5c53d1267f
I20260806 02:50:24.787396 28884 raft_consensus.cc:2726] T 8243a6730a6e4939998608a4e4b63a18 P 8ebd96b903b944f3b06851324b737bd5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260806 02:50:26.123850 21978 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:50:26.124168 21978 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:50:26.124269 21978 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
W20260806 02:50:26.124390 21978 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
F20260806 02:50:26.135315 22000 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260806 02:50:26.135386 21998 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.136080 21985 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1ff877d50df44dccb8be0241b847e8c9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
*** Check failure stack trace: ***
*** Aborted at 1785984626 (unix time) try "date -d @1785984626" if you are using GNU date ***
F20260806 02:50:26.135386 21998 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.136080 21985 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1ff877d50df44dccb8be0241b847e8c9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260806 02:50:26.135386 21998 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.136080 21985 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1ff877d50df44dccb8be0241b847e8c9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.137918 21987 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_cac77bd182314ee2a2068374ddeca8d4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
*** Check failure stack trace: ***
W20260806 02:50:26.139988 21976 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.20.60.61:36797)
W20260806 02:50:26.140368 21976 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.20.60.61:36797)
F20260806 02:50:26.135386 21998 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.136080 21985 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1ff877d50df44dccb8be0241b847e8c9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.137918 21987 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_cac77bd182314ee2a2068374ddeca8d4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.141796 21999 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260806 02:50:26.135386 21998 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.136080 21985 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1ff877d50df44dccb8be0241b847e8c9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.137918 21987 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_cac77bd182314ee2a2068374ddeca8d4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.141796 21999 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.142141 21988 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2b3ca9846e244fc593dc07fd72a58f3f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
*** Check failure stack trace: ***
W20260806 02:50:26.142398 21976 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.20.60.61:36797)
W20260806 02:50:26.143280 21976 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.20.60.61:36797)
F20260806 02:50:26.135386 21998 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.136080 21985 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1ff877d50df44dccb8be0241b847e8c9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.137918 21987 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_cac77bd182314ee2a2068374ddeca8d4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.141796 21999 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.142141 21988 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2b3ca9846e244fc593dc07fd72a58f3f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.143877 22009 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260806 02:50:26.135386 21998 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.136080 21985 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1ff877d50df44dccb8be0241b847e8c9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.137918 21987 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_cac77bd182314ee2a2068374ddeca8d4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.141796 21999 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.142141 21988 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_2b3ca9846e244fc593dc07fd72a58f3f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.143877 22009 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.20.60.62:33227,127.20.60.61:36797,127.20.60.60:42309, num_attempts: 736) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused (error 111)F20260806 02:50:26.144517 22002 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: Client connection negotiation failed: client connection to 127.20.60.61:36797: connect: Connection refused
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e8000050f0) received by PID 20720 (TID 0x7fa4937d8700) from PID 20720; stack trace: ***
@ 0x7fa4a760fa46 google::(anonymous namespace)::FailureSignalHandler() at ??:0
@ 0x7fa4ab751980 (unknown) at ??:0
@ 0x7fa4a493de87 gsignal at ??:0
@ 0x7fa4a493f7f1 abort at ??:0
@ 0x7fa4a76019ad google::LogMessage::Fail() at ??:0
@ 0x7fa4a760488a google::LogMessage::SendToLog() at ??:0
@ 0x7fa4a76014de google::LogMessage::Flush() at ??:0
@ 0x7fa4a7602a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x404bb0 kudu::MasterStressTest::AlterTableThread() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/master-stress-test.cc:329
@ 0x444b44 std::__invoke_impl<>() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/bits/invoke.h:73
@ 0x4449fd std::__invoke<>() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/bits/invoke.h:95
@ 0x4449ba _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:244
@ 0x44496e std::thread::_Invoker<>::operator()() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:253
@ 0x4446a9 std::thread::_State_impl<>::_M_run() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:196
@ 0x7fa4b008da4f execute_native_thread_routine at ??:0
@ 0x7fa4ab7466db start_thread at ??:0
@ 0x7fa4a4a2061f clone at ??:0