Note: This is test shard 3 of 3.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/2
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260929 20:47:24.392896 388 test_util.cc:268] Using random seed: 1244769684
2026-09-29T20:47:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T20:47:24Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskXtbUDl/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskXtbUDl/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/security.properties
2026-09-29 20:47:32: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskXtbUDl/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskXtbUDl/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260929 20:47:40.987845 388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
/tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.62:46175
--webserver_interface=127.0.97.62
--webserver_port=0
--builtin_ntp_servers=127.0.97.20:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260929 20:47:41.329195 630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 20:47:41.329499 630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 20:47:41.329633 630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 20:47:41.337416 630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 20:47:41.337529 630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 20:47:41.337599 630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 20:47:41.337706 630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 20:47:41.347039 630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.97.20:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.62:46175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.97.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 19:57:29 UTC on ccycloud.quasar-xxilbq.root.comops.site
build id 4681269
ASAN enabled
I20260929 20:47:41.348448 630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 20:47:41.350308 630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 20:47:41.363080 635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:41.363049 636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:41.363394 638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 20:47:41.363927 630 server_base.cc:1034] running on GCE node
I20260929 20:47:41.364858 630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 20:47:41.366324 630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 20:47:41.367573 630 hybrid_clock.cc:648] HybridClock initialized: now 1790714861367465 us; error 88 us; skew 500 ppm
I20260929 20:47:41.367941 630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 20:47:41.370329 630 webserver.cc:460] Webserver started at http://127.0.97.62:40179/ using document root <none> and password file <none>
I20260929 20:47:41.371047 630 fs_manager.cc:359] Metadata directory not provided
I20260929 20:47:41.371186 630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 20:47:41.371501 630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 20:47:41.374289 630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/instance:
uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8"
format_stamp: "Formatted at 2026-09-29 20:47:41 on dist-test-slave-j0bz"
I20260929 20:47:41.375065 630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal/instance:
uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8"
format_stamp: "Formatted at 2026-09-29 20:47:41 on dist-test-slave-j0bz"
I20260929 20:47:41.380434 630 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 20:47:41.384043 644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 20:47:41.385679 630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 20:47:41.385905 630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data,/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8"
format_stamp: "Formatted at 2026-09-29 20:47:41 on dist-test-slave-j0bz"
I20260929 20:47:41.386116 630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 20:47:41.408898 630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 20:47:41.409765 630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 20:47:41.410090 630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 20:47:41.429323 630 rpc_server.cc:307] RPC server started. Bound to: 127.0.97.62:46175
I20260929 20:47:41.429414 695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.97.62:46175 every 8 connection(s)
I20260929 20:47:41.430761 630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
I20260929 20:47:41.434935 696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:41.436677 388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu as pid 630
I20260929 20:47:41.436967 388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal/instance
I20260929 20:47:41.440148 388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
/tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.61:34103
--webserver_interface=127.0.97.61
--webserver_port=0
--builtin_ntp_servers=127.0.97.20:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260929 20:47:41.444376 696 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } has no permanent_uuid. Determining permanent_uuid...
I20260929 20:47:41.462548 696 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } has no permanent_uuid. Determining permanent_uuid...
W20260929 20:47:41.463754 645 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
W20260929 20:47:41.465345 696 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.97.61:34103: Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
I20260929 20:47:41.514756 696 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } attempt: 1
W20260929 20:47:41.516789 696 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.97.61:34103: Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
I20260929 20:47:41.585171 696 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } attempt: 2
W20260929 20:47:41.587255 696 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.97.61:34103: Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
I20260929 20:47:41.678597 696 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } attempt: 3
W20260929 20:47:41.680754 696 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.97.61:34103: Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
W20260929 20:47:41.795634 697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 20:47:41.795964 697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 20:47:41.796041 697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 20:47:41.803815 697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 20:47:41.803941 697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 20:47:41.803982 697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 20:47:41.804031 697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 20:47:41.814213 697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.97.20:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.61:34103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.97.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 19:57:29 UTC on ccycloud.quasar-xxilbq.root.comops.site
build id 4681269
ASAN enabled
I20260929 20:47:41.815572 697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 20:47:41.817430 697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 20:47:41.824060 696 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } attempt: 4
W20260929 20:47:41.826062 696 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.97.61:34103: Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
W20260929 20:47:41.830861 705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:41.830940 706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:41.830911 708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 20:47:41.832338 697 server_base.cc:1034] running on GCE node
I20260929 20:47:41.833271 697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 20:47:41.834729 697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 20:47:41.836051 697 hybrid_clock.cc:648] HybridClock initialized: now 1790714861835974 us; error 41 us; skew 500 ppm
I20260929 20:47:41.836468 697 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 20:47:41.839182 697 webserver.cc:460] Webserver started at http://127.0.97.61:36563/ using document root <none> and password file <none>
I20260929 20:47:41.839922 697 fs_manager.cc:359] Metadata directory not provided
I20260929 20:47:41.840070 697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 20:47:41.840391 697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 20:47:41.843146 697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data/instance:
uuid: "cc99fb1b059842088a50ba9c56fda6c1"
format_stamp: "Formatted at 2026-09-29 20:47:41 on dist-test-slave-j0bz"
I20260929 20:47:41.843952 697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/wal/instance:
uuid: "cc99fb1b059842088a50ba9c56fda6c1"
format_stamp: "Formatted at 2026-09-29 20:47:41 on dist-test-slave-j0bz"
I20260929 20:47:41.849558 697 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260929 20:47:41.853654 714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 20:47:41.855127 697 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.001s
I20260929 20:47:41.855330 697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data,/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/wal
uuid: "cc99fb1b059842088a50ba9c56fda6c1"
format_stamp: "Formatted at 2026-09-29 20:47:41 on dist-test-slave-j0bz"
I20260929 20:47:41.855540 697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 20:47:41.877856 697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 20:47:41.878737 697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 20:47:41.879055 697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 20:47:41.900110 697 rpc_server.cc:307] RPC server started. Bound to: 127.0.97.61:34103
I20260929 20:47:41.900142 765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.97.61:34103 every 8 connection(s)
I20260929 20:47:41.901551 697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data/info.pb
I20260929 20:47:41.905210 766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:41.910441 388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu as pid 697
I20260929 20:47:41.910645 388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/wal/instance
I20260929 20:47:41.911635 388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
/tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.60:45599
--webserver_interface=127.0.97.60
--webserver_port=0
--builtin_ntp_servers=127.0.97.20:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260929 20:47:41.911576 766 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } has no permanent_uuid. Determining permanent_uuid...
I20260929 20:47:41.927731 766 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } has no permanent_uuid. Determining permanent_uuid...
I20260929 20:47:41.939538 766 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } has no permanent_uuid. Determining permanent_uuid...
W20260929 20:47:41.941002 717 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:47:41.942598 766 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.97.60:45599: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:47:41.991973 766 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } attempt: 1
W20260929 20:47:41.994899 766 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.97.60:45599: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:47:42.063190 766 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } attempt: 2
W20260929 20:47:42.065209 766 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.97.60:45599: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:47:42.125341 696 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } attempt: 5
I20260929 20:47:42.133564 696 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } has no permanent_uuid. Determining permanent_uuid...
W20260929 20:47:42.135622 696 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.97.60:45599: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:47:42.156484 766 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } attempt: 3
W20260929 20:47:42.158468 766 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.97.60:45599: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:47:42.186908 696 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } attempt: 1
W20260929 20:47:42.189100 696 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.97.60:45599: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:47:42.257369 696 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } attempt: 2
W20260929 20:47:42.259524 696 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.97.60:45599: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:47:42.269852 767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 20:47:42.270148 767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 20:47:42.270231 767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 20:47:42.277642 767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 20:47:42.277750 767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 20:47:42.277791 767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 20:47:42.277837 767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 20:47:42.287580 767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.97.20:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.60:45599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.97.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 19:57:29 UTC on ccycloud.quasar-xxilbq.root.comops.site
build id 4681269
ASAN enabled
I20260929 20:47:42.289074 767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 20:47:42.290877 767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 20:47:42.301888 766 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } attempt: 4
W20260929 20:47:42.302922 775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:42.303115 778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:42.303259 776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:42.304483 766 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.97.60:45599: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:47:42.305215 767 server_base.cc:1034] running on GCE node
I20260929 20:47:42.305989 767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 20:47:42.307438 767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 20:47:42.308630 767 hybrid_clock.cc:648] HybridClock initialized: now 1790714862308557 us; error 49 us; skew 500 ppm
I20260929 20:47:42.308926 767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 20:47:42.311590 767 webserver.cc:460] Webserver started at http://127.0.97.60:43799/ using document root <none> and password file <none>
I20260929 20:47:42.312245 767 fs_manager.cc:359] Metadata directory not provided
I20260929 20:47:42.312338 767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 20:47:42.312667 767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 20:47:42.315294 767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data/instance:
uuid: "bfbad3ef4593415191fa0d993866d0ea"
format_stamp: "Formatted at 2026-09-29 20:47:42 on dist-test-slave-j0bz"
I20260929 20:47:42.316035 767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/wal/instance:
uuid: "bfbad3ef4593415191fa0d993866d0ea"
format_stamp: "Formatted at 2026-09-29 20:47:42 on dist-test-slave-j0bz"
I20260929 20:47:42.321483 767 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260929 20:47:42.325031 784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 20:47:42.326407 767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 20:47:42.326557 767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data,/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/wal
uuid: "bfbad3ef4593415191fa0d993866d0ea"
format_stamp: "Formatted at 2026-09-29 20:47:42 on dist-test-slave-j0bz"
I20260929 20:47:42.326726 767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 20:47:42.346692 767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 20:47:42.347500 767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 20:47:42.347748 767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 20:47:42.365849 696 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } attempt: 3
W20260929 20:47:42.367936 696 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.97.60:45599: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:47:42.369503 767 rpc_server.cc:307] RPC server started. Bound to: 127.0.97.60:45599
I20260929 20:47:42.369518 835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.97.60:45599 every 8 connection(s)
I20260929 20:47:42.370855 767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data/info.pb
I20260929 20:47:42.374804 836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:42.379560 388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu as pid 767
I20260929 20:47:42.379760 388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/wal/instance
I20260929 20:47:42.381011 836 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } has no permanent_uuid. Determining permanent_uuid...
I20260929 20:47:42.397588 836 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } has no permanent_uuid. Determining permanent_uuid...
I20260929 20:47:42.407292 836 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } has no permanent_uuid. Determining permanent_uuid...
I20260929 20:47:42.422271 836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: Bootstrap starting.
I20260929 20:47:42.425969 836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:42.426975 836 log.cc:824] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: Log is configured to *not* fsync() on all Append() calls
I20260929 20:47:42.429489 836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: No bootstrap required, opened a new log
I20260929 20:47:42.438462 836 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:42.439003 836 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:42.439132 836 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfbad3ef4593415191fa0d993866d0ea, State: Initialized, Role: FOLLOWER
I20260929 20:47:42.439817 836 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:42.441542 844 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:42.441965 844 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:42.442617 836 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: configured and running, proceeding with master startup.
W20260929 20:47:42.462683 858 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 20:47:42.462958 858 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 20:47:42.453362 767 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: OK, instance_id { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" instance_seqno: 1790714861423741 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 20:47:42.465345 767 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:47:42.545220 696 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } attempt: 4
I20260929 20:47:42.557624 696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Bootstrap starting.
I20260929 20:47:42.562783 696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:42.564249 696 log.cc:824] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Log is configured to *not* fsync() on all Append() calls
I20260929 20:47:42.567476 696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: No bootstrap required, opened a new log
I20260929 20:47:42.579080 696 raft_consensus.cc:348] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:42.579563 696 raft_consensus.cc:374] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:42.579690 696 raft_consensus.cc:729] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32aaf3821ff4ad09bcbfe11ff2e03c8, State: Initialized, Role: FOLLOWER
I20260929 20:47:42.580389 696 consensus_queue.cc:260] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:42.581344 862 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:42.581694 862 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:42.582803 696 sys_catalog.cc:564] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: configured and running, proceeding with master startup.
W20260929 20:47:42.600606 790 tablet.cc:2369] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 20:47:42.603829 766 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } attempt: 5
I20260929 20:47:42.612054 862 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:42.612382 862 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:42.614159 862 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
I20260929 20:47:42.616295 811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfbad3ef4593415191fa0d993866d0ea" is_pre_election: true
I20260929 20:47:42.617009 811 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b32aaf3821ff4ad09bcbfe11ff2e03c8 in term 0.
I20260929 20:47:42.617972 647 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea; no voters:
I20260929 20:47:42.622370 862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 20:47:42.622558 862 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:47:42.622653 862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:42.628244 862 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:42.629316 862 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 1 election: Requested vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
I20260929 20:47:42.594710 630 master_runner.cc:186] Error getting master registration for 127.0.97.61:34103: OK, instance_id { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" instance_seqno: 1790714861894233 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 20:47:42.632750 811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfbad3ef4593415191fa0d993866d0ea"
I20260929 20:47:42.632984 811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:42.633024 630 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 20:47:42.635926 876 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 20:47:42.636044 876 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 20:47:42.636330 811 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b32aaf3821ff4ad09bcbfe11ff2e03c8 in term 1.
I20260929 20:47:42.636901 647 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea; no voters:
I20260929 20:47:42.637650 862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 20:47:42.639322 766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: Bootstrap starting.
I20260929 20:47:42.646086 766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:42.648541 766 log.cc:824] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: Log is configured to *not* fsync() on all Append() calls
I20260929 20:47:42.655525 862 raft_consensus.cc:686] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 1 LEADER]: Becoming Leader. State: Replica: b32aaf3821ff4ad09bcbfe11ff2e03c8, State: Running, Role: LEADER
I20260929 20:47:42.656379 862 consensus_queue.cc:237] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:42.659392 766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: No bootstrap required, opened a new log
W20260929 20:47:42.662925 650 tablet.cc:2369] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 20:47:42.670408 879 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b32aaf3821ff4ad09bcbfe11ff2e03c8. Latest consensus state: current_term: 1 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:42.670907 879 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: LEADER
I20260929 20:47:42.679288 882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 20:47:42.680037 766 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:42.680524 766 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:42.680644 766 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc99fb1b059842088a50ba9c56fda6c1, State: Initialized, Role: FOLLOWER
I20260929 20:47:42.681339 766 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:42.682811 883 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:42.683657 883 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:42.684260 766 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 20:47:42.684226 741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc99fb1b059842088a50ba9c56fda6c1" is_pre_election: true
I20260929 20:47:42.684930 741 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b32aaf3821ff4ad09bcbfe11ff2e03c8 in term 0.
I20260929 20:47:42.685478 740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc99fb1b059842088a50ba9c56fda6c1"
I20260929 20:47:42.685815 740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:42.687040 882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 20:47:42.690477 740 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b32aaf3821ff4ad09bcbfe11ff2e03c8 in term 1.
I20260929 20:47:42.714578 811 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 1 FOLLOWER]: Refusing update from remote peer b32aaf3821ff4ad09bcbfe11ff2e03c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 20:47:42.716815 740 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 1 FOLLOWER]: Refusing update from remote peer b32aaf3821ff4ad09bcbfe11ff2e03c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 20:47:42.717042 879 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 20:47:42.718094 879 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 20:47:42.762152 883 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b32aaf3821ff4ad09bcbfe11ff2e03c8. Latest consensus state: current_term: 1 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:42.762583 883 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:42.765800 885 mvcc.cc:205] Tried to move back new op lower bound from 7334768077648265216 to 7334768077446852608. Current Snapshot: MvccSnapshot[applied={T|T < 7334768077648265216}]
I20260929 20:47:42.766777 844 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader b32aaf3821ff4ad09bcbfe11ff2e03c8. Latest consensus state: current_term: 1 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:42.767043 844 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:42.770663 889 mvcc.cc:205] Tried to move back new op lower bound from 7334768077648265216 to 7334768077446852608. Current Snapshot: MvccSnapshot[applied={T|T < 7334768077648265216}]
I20260929 20:47:42.771543 844 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:42.771780 844 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:42.774816 888 mvcc.cc:205] Tried to move back new op lower bound from 7334768077648265216 to 7334768077446852608. Current Snapshot: MvccSnapshot[applied={T|T < 7334768077648265216}]
I20260929 20:47:42.778187 883 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:42.778564 883 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:42.779801 882 catalog_manager.cc:1347] Generated new cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:47:42.779932 882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 20:47:42.784925 879 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:42.784857 862 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:42.785265 879 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: LEADER
I20260929 20:47:42.785341 862 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: LEADER
I20260929 20:47:42.805778 907 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 20:47:42.808252 882 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 20:47:42.810914 882 catalog_manager.cc:1504] Loading token signing keys...
I20260929 20:47:42.828609 882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Generated new TSK 0
I20260929 20:47:42.832347 882 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 20:47:42.833917 882 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260929 20:47:42.835188 882 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260929 20:47:42.854908 388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
/tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.97.1:0
--local_ip_for_outbound_sockets=127.0.97.1
--webserver_interface=127.0.97.1
--webserver_port=0
--tserver_master_addrs=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--builtin_ntp_servers=127.0.97.20:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20260929 20:47:43.309057 908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 20:47:43.309468 908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 20:47:43.309562 908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 20:47:43.316989 908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 20:47:43.317173 908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.97.1
I20260929 20:47:43.327636 908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.97.20:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.97.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.97.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.97.1
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 19:57:29 UTC on ccycloud.quasar-xxilbq.root.comops.site
build id 4681269
ASAN enabled
I20260929 20:47:43.328922 912 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:47:43.329142 912 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:43.329506 908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 20:47:43.330525 912 leader_election.cc:288] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103)
I20260929 20:47:43.331095 740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfbad3ef4593415191fa0d993866d0ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cc99fb1b059842088a50ba9c56fda6c1" is_pre_election: true
I20260929 20:47:43.331143 671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfbad3ef4593415191fa0d993866d0ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" is_pre_election: true
I20260929 20:47:43.331926 908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 20:47:43.332069 786 leader_election.cc:302] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfbad3ef4593415191fa0d993866d0ea; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:47:43.332448 912 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260929 20:47:43.346688 914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:43.346606 915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:43.346653 917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 20:47:43.347800 908 server_base.cc:1034] running on GCE node
I20260929 20:47:43.348587 908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 20:47:43.349622 908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 20:47:43.350881 908 hybrid_clock.cc:648] HybridClock initialized: now 1790714863350779 us; error 32 us; skew 500 ppm
I20260929 20:47:43.351249 908 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 20:47:43.353660 908 webserver.cc:460] Webserver started at http://127.0.97.1:44353/ using document root <none> and password file <none>
I20260929 20:47:43.354383 908 fs_manager.cc:359] Metadata directory not provided
I20260929 20:47:43.354512 908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 20:47:43.354835 908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 20:47:43.357501 908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/data/instance:
uuid: "d1f79a0bcbdb42fe89e7812efeae1a86"
format_stamp: "Formatted at 2026-09-29 20:47:43 on dist-test-slave-j0bz"
I20260929 20:47:43.358289 908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/wal/instance:
uuid: "d1f79a0bcbdb42fe89e7812efeae1a86"
format_stamp: "Formatted at 2026-09-29 20:47:43 on dist-test-slave-j0bz"
I20260929 20:47:43.363811 908 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 20:47:43.367501 923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 20:47:43.369256 908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 20:47:43.369467 908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/wal
uuid: "d1f79a0bcbdb42fe89e7812efeae1a86"
format_stamp: "Formatted at 2026-09-29 20:47:43 on dist-test-slave-j0bz"
I20260929 20:47:43.369663 908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 20:47:43.392948 908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 20:47:43.393807 908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 20:47:43.394119 908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 20:47:43.395042 908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 20:47:43.396612 908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 20:47:43.396708 908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 20:47:43.396867 908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 20:47:43.396938 908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 20:47:43.434412 908 rpc_server.cc:307] RPC server started. Bound to: 127.0.97.1:45657
I20260929 20:47:43.434513 1035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.97.1:45657 every 8 connection(s)
I20260929 20:47:43.436131 908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/data/info.pb
I20260929 20:47:43.441304 388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu as pid 908
I20260929 20:47:43.441506 388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/wal/instance
I20260929 20:47:43.447352 388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
/tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.97.2:0
--local_ip_for_outbound_sockets=127.0.97.2
--webserver_interface=127.0.97.2
--webserver_port=0
--tserver_master_addrs=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--builtin_ntp_servers=127.0.97.20:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260929 20:47:43.466535 858 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:47:43.466691 858 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: loading cluster ID for follower catalog manager: success
I20260929 20:47:43.471683 858 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: acquiring CA information for follower catalog manager: success
I20260929 20:47:43.487306 858 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 20:47:43.491577 1037 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:47:43.491948 1037 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:43.493022 1037 heartbeater.cc:511] Master 127.0.97.62:46175 requested a full tablet report, sending...
I20260929 20:47:43.495486 661 ts_manager.cc:194] Registered new tserver with Master: d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657)
I20260929 20:47:43.497886 1036 heartbeater.cc:347] Connected to a master server at 127.0.97.60:45599
I20260929 20:47:43.498075 1036 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:43.498328 661 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.97.1:58905
I20260929 20:47:43.498704 1036 heartbeater.cc:511] Master 127.0.97.60:45599 requested a full tablet report, sending...
I20260929 20:47:43.501214 801 ts_manager.cc:194] Registered new tserver with Master: d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657)
I20260929 20:47:43.506016 1038 heartbeater.cc:347] Connected to a master server at 127.0.97.61:34103
I20260929 20:47:43.506222 1038 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:43.506613 1038 heartbeater.cc:511] Master 127.0.97.61:34103 requested a full tablet report, sending...
I20260929 20:47:43.508481 731 ts_manager.cc:194] Registered new tserver with Master: d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657)
I20260929 20:47:43.810972 907 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:47:43.811124 907 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: loading cluster ID for follower catalog manager: success
I20260929 20:47:43.815191 907 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: acquiring CA information for follower catalog manager: success
I20260929 20:47:43.817999 907 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 20:47:43.822340 1048 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:47:43.822618 1048 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:43.824843 671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc99fb1b059842088a50ba9c56fda6c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" is_pre_election: true
I20260929 20:47:43.825670 1048 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
I20260929 20:47:43.826265 810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc99fb1b059842088a50ba9c56fda6c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bfbad3ef4593415191fa0d993866d0ea" is_pre_election: true
I20260929 20:47:43.826939 717 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:47:43.827360 1048 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260929 20:47:43.855648 1043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 20:47:43.855942 1043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 20:47:43.856091 1043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 20:47:43.864339 1043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 20:47:43.864588 1043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.97.2
I20260929 20:47:43.874040 1052 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:47:43.874306 1052 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:43.875221 1043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.97.20:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.97.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.97.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.97.2
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 19:57:29 UTC on ccycloud.quasar-xxilbq.root.comops.site
build id 4681269
ASAN enabled
I20260929 20:47:43.875686 671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfbad3ef4593415191fa0d993866d0ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" is_pre_election: true
I20260929 20:47:43.875800 740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfbad3ef4593415191fa0d993866d0ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cc99fb1b059842088a50ba9c56fda6c1" is_pre_election: true
I20260929 20:47:43.876061 740 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfbad3ef4593415191fa0d993866d0ea in term 1.
I20260929 20:47:43.876574 785 leader_election.cc:302] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bfbad3ef4593415191fa0d993866d0ea, cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8
I20260929 20:47:43.877053 1043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 20:47:43.877104 1052 leader_election.cc:288] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103)
I20260929 20:47:43.877251 1053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 20:47:43.877383 1053 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:47:43.877516 1053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 1 FOLLOWER]: Advancing to term 2
I20260929 20:47:43.879436 1043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 20:47:43.880455 1053 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:43.881248 1053 leader_election.cc:288] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 2 election: Requested vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103)
I20260929 20:47:43.881805 671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfbad3ef4593415191fa0d993866d0ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8"
I20260929 20:47:43.882652 740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfbad3ef4593415191fa0d993866d0ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cc99fb1b059842088a50ba9c56fda6c1"
I20260929 20:47:43.882922 740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 1 FOLLOWER]: Advancing to term 2
I20260929 20:47:43.886078 740 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfbad3ef4593415191fa0d993866d0ea in term 2.
I20260929 20:47:43.886611 785 leader_election.cc:302] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bfbad3ef4593415191fa0d993866d0ea, cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8
I20260929 20:47:43.887113 1053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 2 FOLLOWER]: Leader election won for term 2
I20260929 20:47:43.887324 1053 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 2 LEADER]: Becoming Leader. State: Replica: bfbad3ef4593415191fa0d993866d0ea, State: Running, Role: LEADER
I20260929 20:47:43.887969 1053 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
W20260929 20:47:43.899714 1055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:43.900802 1056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 20:47:43.900995 1052 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfbad3ef4593415191fa0d993866d0ea. Latest consensus state: current_term: 2 leader_uuid: "bfbad3ef4593415191fa0d993866d0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:43.901320 1052 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: This master's current role is: LEADER
I20260929 20:47:43.909112 1059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 20:47:43.911589 1059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 20:47:43.912739 1059 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:47:43.912909 1059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 20:47:43.914458 1059 catalog_manager.cc:1504] Loading token signing keys...
I20260929 20:47:43.915508 1059 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: Loaded TSK: 0
I20260929 20:47:43.916764 1059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 20:47:43.916770 1043 server_base.cc:1034] running on GCE node
I20260929 20:47:43.917868 1059 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
W20260929 20:47:43.917948 1060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 20:47:43.918797 1043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 20:47:43.919332 1059 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260929 20:47:43.920765 1043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 20:47:43.920878 740 raft_consensus.cc:1226] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 2 FOLLOWER]: Rejecting Update request from peer b32aaf3821ff4ad09bcbfe11ff2e03c8 for earlier term 1. Current term is 2. Ops: []
I20260929 20:47:43.921629 1062 consensus_queue.cc:1046] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260929 20:47:43.922099 1043 hybrid_clock.cc:648] HybridClock initialized: now 1790714863922002 us; error 62 us; skew 500 ppm
I20260929 20:47:43.922435 1043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 20:47:43.923170 1064 raft_consensus.cc:3032] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260929 20:47:43.923306 1064 raft_consensus.cc:729] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b32aaf3821ff4ad09bcbfe11ff2e03c8, State: Running, Role: LEADER
I20260929 20:47:43.923692 1064 consensus_queue.cc:260] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:43.924331 1064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 1 FOLLOWER]: Advancing to term 2
I20260929 20:47:43.926182 1043 webserver.cc:460] Webserver started at http://127.0.97.2:42245/ using document root <none> and password file <none>
I20260929 20:47:43.927069 1043 fs_manager.cc:359] Metadata directory not provided
I20260929 20:47:43.927208 1043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 20:47:43.927541 1043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 20:47:43.930478 1043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/data/instance:
uuid: "cac815b9c12e40c6b63d4af82248f298"
format_stamp: "Formatted at 2026-09-29 20:47:43 on dist-test-slave-j0bz"
I20260929 20:47:43.931388 1043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/wal/instance:
uuid: "cac815b9c12e40c6b63d4af82248f298"
format_stamp: "Formatted at 2026-09-29 20:47:43 on dist-test-slave-j0bz"
I20260929 20:47:43.937474 1043 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260929 20:47:43.941206 1068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 20:47:43.942917 1043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260929 20:47:43.943104 1043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/wal
uuid: "cac815b9c12e40c6b63d4af82248f298"
format_stamp: "Formatted at 2026-09-29 20:47:43 on dist-test-slave-j0bz"
I20260929 20:47:43.943282 1043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 20:47:43.958495 1043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 20:47:43.959520 1043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 20:47:43.959813 1043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 20:47:43.961452 1043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 20:47:43.964102 1043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 20:47:43.964233 1043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 20:47:43.964370 1043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 20:47:43.964421 1043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 20:47:44.002297 1037 heartbeater.cc:503] Master 127.0.97.62:46175 was elected leader, sending a full tablet report...
I20260929 20:47:44.009898 1043 rpc_server.cc:307] RPC server started. Bound to: 127.0.97.2:41081
I20260929 20:47:44.009956 1180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.97.2:41081 every 8 connection(s)
I20260929 20:47:44.011596 1043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/data/info.pb
I20260929 20:47:44.017653 388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu as pid 1043
I20260929 20:47:44.017903 388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/wal/instance
I20260929 20:47:44.029102 388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
/tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.97.3:0
--local_ip_for_outbound_sockets=127.0.97.3
--webserver_interface=127.0.97.3
--webserver_port=0
--tserver_master_addrs=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--builtin_ntp_servers=127.0.97.20:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260929 20:47:44.097270 1183 heartbeater.cc:347] Connected to a master server at 127.0.97.61:34103
I20260929 20:47:44.097656 1183 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:44.098591 1183 heartbeater.cc:511] Master 127.0.97.61:34103 requested a full tablet report, sending...
I20260929 20:47:44.101189 731 ts_manager.cc:194] Registered new tserver with Master: cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:44.104883 1181 heartbeater.cc:347] Connected to a master server at 127.0.97.60:45599
I20260929 20:47:44.105050 1181 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:44.105531 1181 heartbeater.cc:511] Master 127.0.97.60:45599 requested a full tablet report, sending...
I20260929 20:47:44.106596 801 ts_manager.cc:194] Registered new tserver with Master: cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:44.107270 1182 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:47:44.107393 1182 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:44.107795 1182 heartbeater.cc:511] Master 127.0.97.62:46175 requested a full tablet report, sending...
I20260929 20:47:44.108299 801 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.97.2:34593
I20260929 20:47:44.108989 661 ts_manager.cc:194] Registered new tserver with Master: cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:44.300690 740 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 2 FOLLOWER]: Refusing update from remote peer bfbad3ef4593415191fa0d993866d0ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260929 20:47:44.301416 1052 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260929 20:47:44.303857 1048 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfbad3ef4593415191fa0d993866d0ea. Latest consensus state: current_term: 2 leader_uuid: "bfbad3ef4593415191fa0d993866d0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:44.304195 1048 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:44.306825 1053 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "bfbad3ef4593415191fa0d993866d0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:44.307227 1053 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: This master's current role is: LEADER
I20260929 20:47:44.308773 671 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 2 FOLLOWER]: Refusing update from remote peer bfbad3ef4593415191fa0d993866d0ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260929 20:47:44.308866 1048 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "bfbad3ef4593415191fa0d993866d0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:44.309093 1048 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:44.309605 1053 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260929 20:47:44.313258 1062 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "bfbad3ef4593415191fa0d993866d0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:44.313568 1062 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:44.313982 1064 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfbad3ef4593415191fa0d993866d0ea. Latest consensus state: current_term: 2 leader_uuid: "bfbad3ef4593415191fa0d993866d0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:44.314150 1064 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 20:47:44.453181 1186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 20:47:44.453521 1186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 20:47:44.453677 1186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 20:47:44.462083 1186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 20:47:44.462333 1186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.97.3
I20260929 20:47:44.472330 1186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.97.20:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.97.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.97.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.97.3
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 19:57:29 UTC on ccycloud.quasar-xxilbq.root.comops.site
build id 4681269
ASAN enabled
I20260929 20:47:44.473785 1186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 20:47:44.476022 1186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 20:47:44.491078 1202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:44.491446 1201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:44.491163 1204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 20:47:44.492286 1186 server_base.cc:1034] running on GCE node
I20260929 20:47:44.492962 1186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 20:47:44.493920 1186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 20:47:44.495188 1186 hybrid_clock.cc:648] HybridClock initialized: now 1790714864495130 us; error 44 us; skew 500 ppm
I20260929 20:47:44.495553 1186 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 20:47:44.498072 1186 webserver.cc:460] Webserver started at http://127.0.97.3:46219/ using document root <none> and password file <none>
I20260929 20:47:44.498818 1186 fs_manager.cc:359] Metadata directory not provided
I20260929 20:47:44.498966 1186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 20:47:44.499295 1186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 20:47:44.502065 1186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/data/instance:
uuid: "1cb32e9690a546149d11339d371c832f"
format_stamp: "Formatted at 2026-09-29 20:47:44 on dist-test-slave-j0bz"
I20260929 20:47:44.502885 1186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/wal/instance:
uuid: "1cb32e9690a546149d11339d371c832f"
format_stamp: "Formatted at 2026-09-29 20:47:44 on dist-test-slave-j0bz"
I20260929 20:47:44.506565 1036 heartbeater.cc:503] Master 127.0.97.60:45599 was elected leader, sending a full tablet report...
I20260929 20:47:44.508785 1186 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 20:47:44.513218 1210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 20:47:44.515022 1186 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260929 20:47:44.515237 1186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/wal
uuid: "1cb32e9690a546149d11339d371c832f"
format_stamp: "Formatted at 2026-09-29 20:47:44 on dist-test-slave-j0bz"
I20260929 20:47:44.515481 1186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 20:47:44.547036 1186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 20:47:44.548048 1186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 20:47:44.548350 1186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 20:47:44.549456 1186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 20:47:44.551005 1186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 20:47:44.551107 1186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 20:47:44.551252 1186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 20:47:44.551331 1186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 20:47:44.591616 1186 rpc_server.cc:307] RPC server started. Bound to: 127.0.97.3:32835
I20260929 20:47:44.591677 1322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.97.3:32835 every 8 connection(s)
I20260929 20:47:44.593012 1186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/data/info.pb
I20260929 20:47:44.598461 388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu as pid 1186
I20260929 20:47:44.598631 388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/wal/instance
I20260929 20:47:44.613154 1181 heartbeater.cc:503] Master 127.0.97.60:45599 was elected leader, sending a full tablet report...
I20260929 20:47:44.615330 1323 heartbeater.cc:347] Connected to a master server at 127.0.97.60:45599
I20260929 20:47:44.615674 1323 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:44.616595 1324 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:47:44.616750 1324 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:44.616829 1323 heartbeater.cc:511] Master 127.0.97.60:45599 requested a full tablet report, sending...
I20260929 20:47:44.617291 1324 heartbeater.cc:511] Master 127.0.97.62:46175 requested a full tablet report, sending...
I20260929 20:47:44.618468 1325 heartbeater.cc:347] Connected to a master server at 127.0.97.61:34103
I20260929 20:47:44.618578 1325 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:44.618643 661 ts_manager.cc:194] Registered new tserver with Master: 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:44.618813 801 ts_manager.cc:194] Registered new tserver with Master: 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:44.618993 1325 heartbeater.cc:511] Master 127.0.97.61:34103 requested a full tablet report, sending...
I20260929 20:47:44.619772 801 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.97.3:33173
I20260929 20:47:44.621397 731 ts_manager.cc:194] Registered new tserver with Master: 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:44.625320 388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 20:47:44.639997 876 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 20:47:44.664386 801 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36350:
name: "default.Table_0b803212f31d45419130e1da10d1a620"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.664397 800 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36350:
name: "default.Table_4cf3799777554d47b5dbb898a6d368b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.668429 799 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36350:
name: "default.Table_5b5c251fd82449869826c01b42a62f31"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.668901 792 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36350:
name: "default.Table_952135bfbb564f519938b6b206b1d463"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.669335 797 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36350:
name: "default.Table_a5aa493381b543419904f1429ffc19f2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.669687 794 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36350:
name: "default.Table_ef3bd0e733d74bf8a3cc021239e8dea5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.670046 795 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36350:
name: "default.Table_d9ce7635a33b4f6ea325f294cc7db677"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.670511 796 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36350:
name: "default.Table_47f3dc18fe564237bda167352f1afa08"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.670675 793 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36350:
name: "default.Table_f02a75c1e46d472d90334b4340d99e7b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.671203 798 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36350:
name: "default.Table_556595b54fef48f99c72c48e730495e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 20:47:44.677102 800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4cf3799777554d47b5dbb898a6d368b9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:44.680303 801 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_0b803212f31d45419130e1da10d1a620 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:44.680425 799 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_5b5c251fd82449869826c01b42a62f31 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:44.681818 792 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_952135bfbb564f519938b6b206b1d463 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:44.682583 798 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_556595b54fef48f99c72c48e730495e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:44.683522 793 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f02a75c1e46d472d90334b4340d99e7b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:44.684365 796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_47f3dc18fe564237bda167352f1afa08 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:44.685340 795 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d9ce7635a33b4f6ea325f294cc7db677 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:44.686250 794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_ef3bd0e733d74bf8a3cc021239e8dea5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:44.687139 797 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_a5aa493381b543419904f1429ffc19f2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 20:47:44.715034 388 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu pid 767
I20260929 20:47:44.716235 831 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 20:47:44.807428 1048 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bfbad3ef4593415191fa0d993866d0ea)
I20260929 20:47:44.807646 1048 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:44.808444 1048 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
I20260929 20:47:44.809393 671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc99fb1b059842088a50ba9c56fda6c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" is_pre_election: true
I20260929 20:47:44.811251 1064 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bfbad3ef4593415191fa0d993866d0ea)
I20260929 20:47:44.811406 1064 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:44.812005 1064 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
I20260929 20:47:44.812917 740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cc99fb1b059842088a50ba9c56fda6c1" is_pre_election: true
I20260929 20:47:44.813153 740 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b32aaf3821ff4ad09bcbfe11ff2e03c8 in term 2.
I20260929 20:47:44.813768 645 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, cc99fb1b059842088a50ba9c56fda6c1; no voters:
I20260929 20:47:44.814052 1062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260929 20:47:44.814141 1062 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader bfbad3ef4593415191fa0d993866d0ea)
I20260929 20:47:44.814211 1062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 2 FOLLOWER]: Advancing to term 3
I20260929 20:47:44.817085 1062 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:44.819881 1062 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 3 election: Requested vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
I20260929 20:47:44.820530 740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cc99fb1b059842088a50ba9c56fda6c1"
I20260929 20:47:44.820745 740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 2 FOLLOWER]: Advancing to term 3
I20260929 20:47:44.823585 740 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b32aaf3821ff4ad09bcbfe11ff2e03c8 in term 3.
I20260929 20:47:44.825145 645 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, cc99fb1b059842088a50ba9c56fda6c1; no voters:
I20260929 20:47:44.825632 1062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 20:47:44.825997 1062 raft_consensus.cc:686] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 3 LEADER]: Becoming Leader. State: Replica: b32aaf3821ff4ad09bcbfe11ff2e03c8, State: Running, Role: LEADER
I20260929 20:47:44.826305 1062 consensus_queue.cc:237] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:44.827467 1064 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b32aaf3821ff4ad09bcbfe11ff2e03c8. Latest consensus state: current_term: 3 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:44.827706 1064 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: LEADER
I20260929 20:47:44.836339 1371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 20:47:44.838470 1371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 20:47:44.840096 1371 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:47:44.840240 1371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 20:47:44.842041 1371 catalog_manager.cc:1504] Loading token signing keys...
I20260929 20:47:44.843029 1371 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Loaded TSK: 0
I20260929 20:47:44.844018 1371 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 20:47:44.844944 1371 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260929 20:47:44.845897 1371 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260929 20:47:44.893774 809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc99fb1b059842088a50ba9c56fda6c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bfbad3ef4593415191fa0d993866d0ea" is_pre_election: true
I20260929 20:47:44.893870 810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bfbad3ef4593415191fa0d993866d0ea" is_pre_election: true
I20260929 20:47:44.894402 811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "bfbad3ef4593415191fa0d993866d0ea"
I20260929 20:47:44.894846 388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu with pid 767
I20260929 20:47:44.894876 717 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:47:44.895367 1048 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260929 20:47:44.922807 1333 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.97.60:45599)
W20260929 20:47:44.928287 1337 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 20:47:44.924646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.97.60:45599 (local address 127.0.0.1:36360)
0929 20:47:44.925055 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0929 20:47:44.925080 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 20:47:44.925238 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 20:47:44.926698 (+ 1460us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":144}
I20260929 20:47:44.928525 661 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35484:
name: "default.Table_952135bfbb564f519938b6b206b1d463"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.928723 659 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35484:
name: "default.Table_ef3bd0e733d74bf8a3cc021239e8dea5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.928818 660 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35484:
name: "default.Table_f02a75c1e46d472d90334b4340d99e7b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.929513 658 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35484:
name: "default.Table_d9ce7635a33b4f6ea325f294cc7db677"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.929804 657 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35484:
name: "default.Table_0b803212f31d45419130e1da10d1a620"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.930274 656 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35484:
name: "default.Table_4cf3799777554d47b5dbb898a6d368b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.931557 655 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35484:
name: "default.Table_5b5c251fd82449869826c01b42a62f31"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.931918 654 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35484:
name: "default.Table_556595b54fef48f99c72c48e730495e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.932624 652 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35484:
name: "default.Table_47f3dc18fe564237bda167352f1afa08"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:44.933156 653 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35484:
name: "default.Table_a5aa493381b543419904f1429ffc19f2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 20:47:44.937712 659 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_ef3bd0e733d74bf8a3cc021239e8dea5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:44.937712 661 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_952135bfbb564f519938b6b206b1d463 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:44.940017 647 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:47:44.940378 647 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:47:45.016598 1036 heartbeater.cc:644] Failed to heartbeat to 127.0.97.60:45599 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:47:45.116947 1181 heartbeater.cc:644] Failed to heartbeat to 127.0.97.60:45599 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:47:45.122504 1323 heartbeater.cc:503] Master 127.0.97.60:45599 was elected leader, sending a full tablet report...
W20260929 20:47:45.128074 1323 heartbeater.cc:644] Failed to heartbeat to 127.0.97.60:45599 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:47:45.190943 661 catalog_manager.cc:2148] created HMS catalog entry for table [id=f04d505b1ef9453780f36b454bc22f46]
I20260929 20:47:45.195456 740 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 3 FOLLOWER]: Refusing update from remote peer b32aaf3821ff4ad09bcbfe11ff2e03c8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
W20260929 20:47:45.196053 647 consensus_peers.cc:487] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 -> Peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Couldn't send request to peer bfbad3ef4593415191fa0d993866d0ea. Status: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 20:47:45.196100 1064 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260929 20:47:45.203716 1048 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b32aaf3821ff4ad09bcbfe11ff2e03c8. Latest consensus state: current_term: 3 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:45.204030 1048 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:45.213706 1048 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:45.213996 1048 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:45.220184 1062 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:45.220639 1062 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: LEADER
I20260929 20:47:45.229159 661 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35484:
table { table_name: "default.Table_952135bfbb564f519938b6b206b1d463" } new_table_name: "default.Table_c7abc2a81b1e45a682a347c05afa8fc7" modify_external_catalogs: true
I20260929 20:47:45.233450 659 catalog_manager.cc:2148] created HMS catalog entry for table [id=843416a726514cbfb345820f1877b73f]
W20260929 20:47:45.236545 658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d9ce7635a33b4f6ea325f294cc7db677 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:45.236613 653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_a5aa493381b543419904f1429ffc19f2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:45.237620 657 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_0b803212f31d45419130e1da10d1a620 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:45.238616 656 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4cf3799777554d47b5dbb898a6d368b9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:45.239560 660 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f02a75c1e46d472d90334b4340d99e7b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:45.240473 655 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_5b5c251fd82449869826c01b42a62f31 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:45.241397 654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_556595b54fef48f99c72c48e730495e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260929 20:47:45.242702 652 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_47f3dc18fe564237bda167352f1afa08 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 20:47:45.275072 659 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35484:
table { table_name: "default.Table_ef3bd0e733d74bf8a3cc021239e8dea5" } new_table_name: "default.Table_129563e3e25e47aebbdbb8d9df626d4b" modify_external_catalogs: true
I20260929 20:47:45.281162 1257 tablet_service.cc:1467] Processing CreateTablet for tablet 4ac5f47c77a842aca9f996f49f34daf0 (DEFAULT_TABLE table=default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.282141 1258 tablet_service.cc:1467] Processing CreateTablet for tablet 95476e3b89fe467ea3f151400dcc4f76 (DEFAULT_TABLE table=default.table_952135bfbb564f519938b6b206b1d463 [id=f04d505b1ef9453780f36b454bc22f46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.282948 1257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ac5f47c77a842aca9f996f49f34daf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.283970 1258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95476e3b89fe467ea3f151400dcc4f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.290818 971 tablet_service.cc:1467] Processing CreateTablet for tablet 95476e3b89fe467ea3f151400dcc4f76 (DEFAULT_TABLE table=default.table_952135bfbb564f519938b6b206b1d463 [id=f04d505b1ef9453780f36b454bc22f46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.291744 658 catalog_manager.cc:2148] created HMS catalog entry for table [id=274aedc84b104ba38dee5cfff431905a]
I20260929 20:47:45.291498 1116 tablet_service.cc:1467] Processing CreateTablet for tablet 95476e3b89fe467ea3f151400dcc4f76 (DEFAULT_TABLE table=default.table_952135bfbb564f519938b6b206b1d463 [id=f04d505b1ef9453780f36b454bc22f46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.292352 971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95476e3b89fe467ea3f151400dcc4f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.292842 970 tablet_service.cc:1467] Processing CreateTablet for tablet 4ac5f47c77a842aca9f996f49f34daf0 (DEFAULT_TABLE table=default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.292881 1115 tablet_service.cc:1467] Processing CreateTablet for tablet 4ac5f47c77a842aca9f996f49f34daf0 (DEFAULT_TABLE table=default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.293633 1116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95476e3b89fe467ea3f151400dcc4f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.294533 970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ac5f47c77a842aca9f996f49f34daf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.295307 1115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ac5f47c77a842aca9f996f49f34daf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.307119 658 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35484:
table { table_name: "default.Table_d9ce7635a33b4f6ea325f294cc7db677" } new_table_name: "default.Table_d9ff14af9acd4cc4a2a2a7bc7d22d17f" modify_external_catalogs: true
I20260929 20:47:45.308300 1114 tablet_service.cc:1467] Processing CreateTablet for tablet 1982dc006d3c45bcb4f843e7067929ee (DEFAULT_TABLE table=default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.308853 1114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1982dc006d3c45bcb4f843e7067929ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.310173 969 tablet_service.cc:1467] Processing CreateTablet for tablet 1982dc006d3c45bcb4f843e7067929ee (DEFAULT_TABLE table=default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.310710 969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1982dc006d3c45bcb4f843e7067929ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.317721 1256 tablet_service.cc:1467] Processing CreateTablet for tablet 1982dc006d3c45bcb4f843e7067929ee (DEFAULT_TABLE table=default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.317883 1388 tablet_bootstrap.cc:492] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298: Bootstrap starting.
I20260929 20:47:45.318221 1256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1982dc006d3c45bcb4f843e7067929ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.321923 1388 tablet_bootstrap.cc:654] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.323215 1388 log.cc:824] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298: Log is configured to *not* fsync() on all Append() calls
I20260929 20:47:45.338464 653 catalog_manager.cc:2148] created HMS catalog entry for table [id=e7f4bcce9df4454a81f6e98a5b2e69e5]
I20260929 20:47:45.346114 653 master_service.cc:946] ReplaceTablet: received request to replace tablet 4ac5f47c77a842aca9f996f49f34daf0 from {username='slave'} at 127.0.0.1:35484
I20260929 20:47:45.349982 1391 tablet_bootstrap.cc:492] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f: Bootstrap starting.
I20260929 20:47:45.351649 1390 tablet_bootstrap.cc:492] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap starting.
I20260929 20:47:45.356048 653 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Sending DeleteTablet for 3 replicas of tablet 4ac5f47c77a842aca9f996f49f34daf0
I20260929 20:47:45.357453 970 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac5f47c77a842aca9f996f49f34daf0 with delete_type TABLET_DATA_DELETED (replaced by tablet c4a57da4e5c24803b7aa3f61c7191e38) from {username='slave'} at 127.0.0.1:47676
W20260929 20:47:45.359321 645 catalog_manager.cc:4821] TS d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): delete failed for tablet 4ac5f47c77a842aca9f996f49f34daf0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ac5f47c77a842aca9f996f49f34daf0 already in progress: creating tablet
I20260929 20:47:45.360381 653 catalog_manager.cc:6565] ReplaceTablet: tablet 4ac5f47c77a842aca9f996f49f34daf0 deleted and replaced by tablet c4a57da4e5c24803b7aa3f61c7191e38
I20260929 20:47:45.360574 1258 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac5f47c77a842aca9f996f49f34daf0 with delete_type TABLET_DATA_DELETED (replaced by tablet c4a57da4e5c24803b7aa3f61c7191e38) from {username='slave'} at 127.0.0.1:54804
I20260929 20:47:45.360874 653 master_service.cc:946] ReplaceTablet: received request to replace tablet 4ac5f47c77a842aca9f996f49f34daf0 from {username='slave'} at 127.0.0.1:35484
I20260929 20:47:45.361548 1116 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac5f47c77a842aca9f996f49f34daf0 with delete_type TABLET_DATA_DELETED (replaced by tablet c4a57da4e5c24803b7aa3f61c7191e38) from {username='slave'} at 127.0.0.1:35688
I20260929 20:47:45.363322 1388 tablet_bootstrap.cc:492] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298: No bootstrap required, opened a new log
I20260929 20:47:45.363651 1388 ts_tablet_manager.cc:1397] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298: Time spent bootstrapping tablet: real 0.046s user 0.007s sys 0.009s
I20260929 20:47:45.368476 1116 tablet_service.cc:1467] Processing CreateTablet for tablet 53c949bb99444778a9f15a781596e986 (DEFAULT_TABLE table=default.table_a5aa493381b543419904f1429ffc19f2 [id=e7f4bcce9df4454a81f6e98a5b2e69e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.369241 1257 tablet_service.cc:1467] Processing CreateTablet for tablet 53c949bb99444778a9f15a781596e986 (DEFAULT_TABLE table=default.table_a5aa493381b543419904f1429ffc19f2 [id=e7f4bcce9df4454a81f6e98a5b2e69e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.369220 970 tablet_service.cc:1467] Processing CreateTablet for tablet 53c949bb99444778a9f15a781596e986 (DEFAULT_TABLE table=default.table_a5aa493381b543419904f1429ffc19f2 [id=e7f4bcce9df4454a81f6e98a5b2e69e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.369663 1257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53c949bb99444778a9f15a781596e986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.369688 970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53c949bb99444778a9f15a781596e986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.373090 653 master_service.cc:946] ReplaceTablet: received request to replace tablet 4ac5f47c77a842aca9f996f49f34daf0 from {username='slave'} at 127.0.0.1:35484
I20260929 20:47:45.374917 1116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53c949bb99444778a9f15a781596e986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.378356 971 tablet_service.cc:1467] Processing CreateTablet for tablet c4a57da4e5c24803b7aa3f61c7191e38 (DEFAULT_TABLE table=default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.378787 971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a57da4e5c24803b7aa3f61c7191e38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.378770 653 master_service.cc:946] ReplaceTablet: received request to replace tablet 1982dc006d3c45bcb4f843e7067929ee from {username='slave'} at 127.0.0.1:35484
I20260929 20:47:45.381138 1114 tablet_service.cc:1467] Processing CreateTablet for tablet c4a57da4e5c24803b7aa3f61c7191e38 (DEFAULT_TABLE table=default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260929 20:47:45.381354 646 catalog_manager.cc:4821] TS cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): delete failed for tablet 4ac5f47c77a842aca9f996f49f34daf0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ac5f47c77a842aca9f996f49f34daf0 already in progress: creating tablet
I20260929 20:47:45.381556 1114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a57da4e5c24803b7aa3f61c7191e38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.387705 1258 tablet_service.cc:1467] Processing CreateTablet for tablet c4a57da4e5c24803b7aa3f61c7191e38 (DEFAULT_TABLE table=default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260929 20:47:45.387951 645 catalog_manager.cc:4821] TS 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): delete failed for tablet 4ac5f47c77a842aca9f996f49f34daf0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ac5f47c77a842aca9f996f49f34daf0 already in progress: creating tablet
I20260929 20:47:45.388221 1258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a57da4e5c24803b7aa3f61c7191e38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.391544 653 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Sending DeleteTablet for 3 replicas of tablet 1982dc006d3c45bcb4f843e7067929ee
I20260929 20:47:45.392011 653 catalog_manager.cc:6565] ReplaceTablet: tablet 1982dc006d3c45bcb4f843e7067929ee deleted and replaced by tablet e0bb85fb9e3d49be88b7fa9dccec6cda
I20260929 20:47:45.392333 653 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35484:
table { table_name: "default.Table_a5aa493381b543419904f1429ffc19f2" } new_table_name: "default.Table_4eeaf34187a44f96b98786696ca454c5" modify_external_catalogs: true
I20260929 20:47:45.392426 1115 tablet_service.cc:1514] Processing DeleteTablet for tablet 1982dc006d3c45bcb4f843e7067929ee with delete_type TABLET_DATA_DELETED (replaced by tablet e0bb85fb9e3d49be88b7fa9dccec6cda) from {username='slave'} at 127.0.0.1:35688
W20260929 20:47:45.393193 646 catalog_manager.cc:4821] TS cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): delete failed for tablet 1982dc006d3c45bcb4f843e7067929ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1982dc006d3c45bcb4f843e7067929ee already in progress: creating tablet
I20260929 20:47:45.394461 969 tablet_service.cc:1514] Processing DeleteTablet for tablet 1982dc006d3c45bcb4f843e7067929ee with delete_type TABLET_DATA_DELETED (replaced by tablet e0bb85fb9e3d49be88b7fa9dccec6cda) from {username='slave'} at 127.0.0.1:47676
I20260929 20:47:45.395213 1257 tablet_service.cc:1514] Processing DeleteTablet for tablet 1982dc006d3c45bcb4f843e7067929ee with delete_type TABLET_DATA_DELETED (replaced by tablet e0bb85fb9e3d49be88b7fa9dccec6cda) from {username='slave'} at 127.0.0.1:54804
W20260929 20:47:45.395646 645 catalog_manager.cc:4821] TS d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): delete failed for tablet 1982dc006d3c45bcb4f843e7067929ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1982dc006d3c45bcb4f843e7067929ee already in progress: creating tablet
I20260929 20:47:45.401067 1115 tablet_service.cc:1467] Processing CreateTablet for tablet e0bb85fb9e3d49be88b7fa9dccec6cda (DEFAULT_TABLE table=default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.401573 1115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0bb85fb9e3d49be88b7fa9dccec6cda. 1 dirs total, 0 dirs full, 0 dirs failed
W20260929 20:47:45.404963 645 catalog_manager.cc:4821] TS 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): delete failed for tablet 1982dc006d3c45bcb4f843e7067929ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1982dc006d3c45bcb4f843e7067929ee already in progress: creating tablet
I20260929 20:47:45.406376 1258 tablet_service.cc:1467] Processing CreateTablet for tablet e0bb85fb9e3d49be88b7fa9dccec6cda (DEFAULT_TABLE table=default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.406824 1258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0bb85fb9e3d49be88b7fa9dccec6cda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.406965 657 catalog_manager.cc:2148] created HMS catalog entry for table [id=796a2c5810ac4950a0f9499208fad0e9]
I20260929 20:47:45.409948 1390 tablet_bootstrap.cc:654] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.412762 657 master_service.cc:946] ReplaceTablet: received request to replace tablet 1982dc006d3c45bcb4f843e7067929ee from {username='slave'} at 127.0.0.1:35484
I20260929 20:47:45.414832 657 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35484:
table { table_name: "default.Table_0b803212f31d45419130e1da10d1a620" } new_table_name: "default.Table_71dc24415d9046608c47e70ce90636c4" modify_external_catalogs: true
I20260929 20:47:45.421444 1391 tablet_bootstrap.cc:654] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.423162 1116 tablet_service.cc:1467] Processing CreateTablet for tablet 9201e11b19f84f0db4e2b132a1b60ff8 (DEFAULT_TABLE table=default.table_0b803212f31d45419130e1da10d1a620 [id=796a2c5810ac4950a0f9499208fad0e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.423537 1258 tablet_service.cc:1467] Processing CreateTablet for tablet 9201e11b19f84f0db4e2b132a1b60ff8 (DEFAULT_TABLE table=default.table_0b803212f31d45419130e1da10d1a620 [id=796a2c5810ac4950a0f9499208fad0e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.423840 971 tablet_service.cc:1467] Processing CreateTablet for tablet 9201e11b19f84f0db4e2b132a1b60ff8 (DEFAULT_TABLE table=default.table_0b803212f31d45419130e1da10d1a620 [id=796a2c5810ac4950a0f9499208fad0e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.424017 1258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9201e11b19f84f0db4e2b132a1b60ff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.424263 971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9201e11b19f84f0db4e2b132a1b60ff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.431859 1116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9201e11b19f84f0db4e2b132a1b60ff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.435346 1390 log.cc:824] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86: Log is configured to *not* fsync() on all Append() calls
I20260929 20:47:45.436698 1391 log.cc:824] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f: Log is configured to *not* fsync() on all Append() calls
I20260929 20:47:45.428889 970 tablet_service.cc:1467] Processing CreateTablet for tablet e0bb85fb9e3d49be88b7fa9dccec6cda (DEFAULT_TABLE table=default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.445186 970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0bb85fb9e3d49be88b7fa9dccec6cda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.445196 1388 raft_consensus.cc:348] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.445720 1388 raft_consensus.cc:374] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.445837 1388 raft_consensus.cc:729] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.448830 1388 consensus_queue.cc:260] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.452968 1390 tablet_bootstrap.cc:492] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86: No bootstrap required, opened a new log
I20260929 20:47:45.453398 1390 ts_tablet_manager.cc:1397] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent bootstrapping tablet: real 0.102s user 0.009s sys 0.000s
I20260929 20:47:45.462092 1391 tablet_bootstrap.cc:492] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f: No bootstrap required, opened a new log
I20260929 20:47:45.462450 1391 ts_tablet_manager.cc:1397] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f: Time spent bootstrapping tablet: real 0.113s user 0.009s sys 0.007s
I20260929 20:47:45.469246 1182 heartbeater.cc:503] Master 127.0.97.62:46175 was elected leader, sending a full tablet report...
I20260929 20:47:45.470237 1388 ts_tablet_manager.cc:1428] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298: Time spent starting tablet: real 0.106s user 0.017s sys 0.010s
I20260929 20:47:45.471287 1388 tablet_bootstrap.cc:492] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298: Bootstrap starting.
I20260929 20:47:45.473842 1388 tablet_bootstrap.cc:654] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.478310 1391 raft_consensus.cc:348] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.478785 1391 raft_consensus.cc:374] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.478894 1391 raft_consensus.cc:729] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.479663 1391 consensus_queue.cc:260] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.484421 1390 raft_consensus.cc:348] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.492642 1390 raft_consensus.cc:374] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.492880 1390 raft_consensus.cc:729] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.494161 1390 consensus_queue.cc:260] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.495369 388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
/tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.60:45599
--webserver_interface=127.0.97.60
--webserver_port=43799
--builtin_ntp_servers=127.0.97.20:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260929 20:47:45.498849 1390 ts_tablet_manager.cc:1428] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent starting tablet: real 0.045s user 0.011s sys 0.004s
I20260929 20:47:45.502056 1388 tablet_bootstrap.cc:492] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298: No bootstrap required, opened a new log
I20260929 20:47:45.502223 1388 ts_tablet_manager.cc:1397] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298: Time spent bootstrapping tablet: real 0.031s user 0.011s sys 0.011s
I20260929 20:47:45.502878 1037 heartbeater.cc:503] Master 127.0.97.62:46175 was elected leader, sending a full tablet report...
I20260929 20:47:45.503234 1388 raft_consensus.cc:348] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.503456 1388 raft_consensus.cc:374] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.503545 1388 raft_consensus.cc:729] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.503819 1388 consensus_queue.cc:260] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.504338 1388 ts_tablet_manager.cc:1428] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260929 20:47:45.504637 1388 tablet_bootstrap.cc:492] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298: Bootstrap starting.
I20260929 20:47:45.506384 1388 tablet_bootstrap.cc:654] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.516158 1390 tablet_bootstrap.cc:492] T 1982dc006d3c45bcb4f843e7067929ee P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap starting.
I20260929 20:47:45.518999 1390 tablet_bootstrap.cc:654] T 1982dc006d3c45bcb4f843e7067929ee P d1f79a0bcbdb42fe89e7812efeae1a86: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.523658 1324 heartbeater.cc:503] Master 127.0.97.62:46175 was elected leader, sending a full tablet report...
I20260929 20:47:45.523820 1388 tablet_bootstrap.cc:492] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298: No bootstrap required, opened a new log
I20260929 20:47:45.523999 1388 ts_tablet_manager.cc:1397] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.000s
I20260929 20:47:45.524829 1388 raft_consensus.cc:348] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:45.525084 1388 raft_consensus.cc:374] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.525151 1388 raft_consensus.cc:729] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.523655 1391 ts_tablet_manager.cc:1428] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f: Time spent starting tablet: real 0.061s user 0.023s sys 0.030s
I20260929 20:47:45.525385 1388 consensus_queue.cc:260] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:45.525981 1388 ts_tablet_manager.cc:1428] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 20:47:45.526270 1388 tablet_bootstrap.cc:492] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298: Bootstrap starting.
I20260929 20:47:45.529127 1388 tablet_bootstrap.cc:654] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.530435 1391 tablet_bootstrap.cc:492] T 1982dc006d3c45bcb4f843e7067929ee P 1cb32e9690a546149d11339d371c832f: Bootstrap starting.
I20260929 20:47:45.532269 1391 tablet_bootstrap.cc:654] T 1982dc006d3c45bcb4f843e7067929ee P 1cb32e9690a546149d11339d371c832f: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.544186 1391 tablet_bootstrap.cc:492] T 1982dc006d3c45bcb4f843e7067929ee P 1cb32e9690a546149d11339d371c832f: No bootstrap required, opened a new log
I20260929 20:47:45.544386 1391 ts_tablet_manager.cc:1397] T 1982dc006d3c45bcb4f843e7067929ee P 1cb32e9690a546149d11339d371c832f: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.008s
I20260929 20:47:45.545511 1391 raft_consensus.cc:348] T 1982dc006d3c45bcb4f843e7067929ee P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.545756 1391 raft_consensus.cc:374] T 1982dc006d3c45bcb4f843e7067929ee P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.545862 1391 raft_consensus.cc:729] T 1982dc006d3c45bcb4f843e7067929ee P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.546139 1391 consensus_queue.cc:260] T 1982dc006d3c45bcb4f843e7067929ee P 1cb32e9690a546149d11339d371c832f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.546679 1391 ts_tablet_manager.cc:1428] T 1982dc006d3c45bcb4f843e7067929ee P 1cb32e9690a546149d11339d371c832f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 20:47:45.546988 1391 tablet_bootstrap.cc:492] T 95476e3b89fe467ea3f151400dcc4f76 P 1cb32e9690a546149d11339d371c832f: Bootstrap starting.
I20260929 20:47:45.553864 1390 tablet_bootstrap.cc:492] T 1982dc006d3c45bcb4f843e7067929ee P d1f79a0bcbdb42fe89e7812efeae1a86: No bootstrap required, opened a new log
I20260929 20:47:45.554093 1391 tablet_bootstrap.cc:654] T 95476e3b89fe467ea3f151400dcc4f76 P 1cb32e9690a546149d11339d371c832f: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.554276 1390 ts_tablet_manager.cc:1397] T 1982dc006d3c45bcb4f843e7067929ee P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent bootstrapping tablet: real 0.038s user 0.011s sys 0.005s
I20260929 20:47:45.555768 1390 raft_consensus.cc:348] T 1982dc006d3c45bcb4f843e7067929ee P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.556231 1390 raft_consensus.cc:374] T 1982dc006d3c45bcb4f843e7067929ee P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.556465 1390 raft_consensus.cc:729] T 1982dc006d3c45bcb4f843e7067929ee P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.556964 1390 consensus_queue.cc:260] T 1982dc006d3c45bcb4f843e7067929ee P d1f79a0bcbdb42fe89e7812efeae1a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.560956 1390 ts_tablet_manager.cc:1428] T 1982dc006d3c45bcb4f843e7067929ee P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260929 20:47:45.561311 1390 tablet_bootstrap.cc:492] T 95476e3b89fe467ea3f151400dcc4f76 P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap starting.
I20260929 20:47:45.562343 1388 tablet_bootstrap.cc:492] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298: No bootstrap required, opened a new log
I20260929 20:47:45.562520 1388 ts_tablet_manager.cc:1397] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298: Time spent bootstrapping tablet: real 0.036s user 0.011s sys 0.014s
I20260929 20:47:45.563351 1388 raft_consensus.cc:348] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.563625 1388 raft_consensus.cc:374] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.563730 1388 raft_consensus.cc:729] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.565582 1390 tablet_bootstrap.cc:654] T 95476e3b89fe467ea3f151400dcc4f76 P d1f79a0bcbdb42fe89e7812efeae1a86: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.566440 1388 consensus_queue.cc:260] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.567126 1388 ts_tablet_manager.cc:1428] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260929 20:47:45.567414 1388 tablet_bootstrap.cc:492] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298: Bootstrap starting.
I20260929 20:47:45.569303 1388 tablet_bootstrap.cc:654] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.576004 1397 raft_consensus.cc:482] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:45.576352 1397 raft_consensus.cc:504] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:45.578796 1397 leader_election.cc:288] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657)
I20260929 20:47:45.584656 1391 tablet_bootstrap.cc:492] T 95476e3b89fe467ea3f151400dcc4f76 P 1cb32e9690a546149d11339d371c832f: No bootstrap required, opened a new log
I20260929 20:47:45.584896 1391 ts_tablet_manager.cc:1397] T 95476e3b89fe467ea3f151400dcc4f76 P 1cb32e9690a546149d11339d371c832f: Time spent bootstrapping tablet: real 0.038s user 0.008s sys 0.007s
I20260929 20:47:45.585661 1391 raft_consensus.cc:348] T 95476e3b89fe467ea3f151400dcc4f76 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:45.586014 1391 raft_consensus.cc:374] T 95476e3b89fe467ea3f151400dcc4f76 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.586091 1391 raft_consensus.cc:729] T 95476e3b89fe467ea3f151400dcc4f76 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.586369 1391 consensus_queue.cc:260] T 95476e3b89fe467ea3f151400dcc4f76 P 1cb32e9690a546149d11339d371c832f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:45.586993 1391 ts_tablet_manager.cc:1428] T 95476e3b89fe467ea3f151400dcc4f76 P 1cb32e9690a546149d11339d371c832f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 20:47:45.587332 1391 tablet_bootstrap.cc:492] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f: Bootstrap starting.
I20260929 20:47:45.589108 1391 tablet_bootstrap.cc:654] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f: Neither blocks nor log segments found. Creating new log.
W20260929 20:47:45.594096 1189 tablet.cc:2369] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 20:47:45.598512 1326 tablet.cc:2369] T 1982dc006d3c45bcb4f843e7067929ee P 1cb32e9690a546149d11339d371c832f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 20:47:45.632483 991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95476e3b89fe467ea3f151400dcc4f76" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" is_pre_election: true
I20260929 20:47:45.634313 1397 raft_consensus.cc:482] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:45.634539 1397 raft_consensus.cc:504] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.635273 1397 leader_election.cc:288] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:45.636426 991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ac5f47c77a842aca9f996f49f34daf0" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" is_pre_election: true
I20260929 20:47:45.636446 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95476e3b89fe467ea3f151400dcc4f76" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f" is_pre_election: true
I20260929 20:47:45.636842 991 raft_consensus.cc:2454] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cac815b9c12e40c6b63d4af82248f298 in term 0.
I20260929 20:47:45.637049 1278 raft_consensus.cc:2454] T 95476e3b89fe467ea3f151400dcc4f76 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cac815b9c12e40c6b63d4af82248f298 in term 0.
I20260929 20:47:45.638351 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ac5f47c77a842aca9f996f49f34daf0" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f" is_pre_election: true
I20260929 20:47:45.638600 1278 raft_consensus.cc:2454] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cac815b9c12e40c6b63d4af82248f298 in term 0.
W20260929 20:47:45.639353 1069 leader_election.cc:341] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): Illegal state: must be running to vote when last-logged opid is not known
I20260929 20:47:45.639799 1069 leader_election.cc:302] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cac815b9c12e40c6b63d4af82248f298, d1f79a0bcbdb42fe89e7812efeae1a86; no voters:
I20260929 20:47:45.640372 1397 raft_consensus.cc:2781] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 20:47:45.640652 1397 raft_consensus.cc:482] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:47:45.640867 1397 raft_consensus.cc:3037] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:45.644697 1397 raft_consensus.cc:504] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.645617 1397 leader_election.cc:288] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 election: Requested vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:45.645725 1391 tablet_bootstrap.cc:492] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f: No bootstrap required, opened a new log
I20260929 20:47:45.645901 1391 ts_tablet_manager.cc:1397] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f: Time spent bootstrapping tablet: real 0.059s user 0.002s sys 0.025s
I20260929 20:47:45.646675 991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ac5f47c77a842aca9f996f49f34daf0" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86"
I20260929 20:47:45.646663 1391 raft_consensus.cc:348] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.646927 1391 raft_consensus.cc:374] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.647018 1391 raft_consensus.cc:729] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.646983 991 raft_consensus.cc:3037] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:45.647274 1391 consensus_queue.cc:260] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.647809 1391 ts_tablet_manager.cc:1428] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260929 20:47:45.650059 1391 tablet_bootstrap.cc:492] T c4a57da4e5c24803b7aa3f61c7191e38 P 1cb32e9690a546149d11339d371c832f: Bootstrap starting.
I20260929 20:47:45.651736 1391 tablet_bootstrap.cc:654] T c4a57da4e5c24803b7aa3f61c7191e38 P 1cb32e9690a546149d11339d371c832f: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.655158 991 raft_consensus.cc:2454] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cac815b9c12e40c6b63d4af82248f298 in term 1.
I20260929 20:47:45.657655 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ac5f47c77a842aca9f996f49f34daf0" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f"
I20260929 20:47:45.657941 1278 raft_consensus.cc:3037] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:45.662297 1278 raft_consensus.cc:2454] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cac815b9c12e40c6b63d4af82248f298 in term 1.
I20260929 20:47:45.663074 1069 leader_election.cc:302] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298; no voters: d1f79a0bcbdb42fe89e7812efeae1a86
I20260929 20:47:45.664003 1069 leader_election.cc:302] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298; no voters:
I20260929 20:47:45.664294 1397 raft_consensus.cc:2781] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 20:47:45.664762 1397 raft_consensus.cc:482] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:47:45.664940 1397 raft_consensus.cc:3037] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:45.666443 1412 raft_consensus.cc:2781] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 20:47:45.669634 1397 raft_consensus.cc:504] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:45.670558 1397 leader_election.cc:288] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 election: Requested vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657)
I20260929 20:47:45.672257 1412 raft_consensus.cc:686] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [term 1 LEADER]: Becoming Leader. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Running, Role: LEADER
I20260929 20:47:45.674366 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95476e3b89fe467ea3f151400dcc4f76" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f"
I20260929 20:47:45.674801 1278 raft_consensus.cc:3037] T 95476e3b89fe467ea3f151400dcc4f76 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:45.679335 1278 raft_consensus.cc:2454] T 95476e3b89fe467ea3f151400dcc4f76 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cac815b9c12e40c6b63d4af82248f298 in term 1.
I20260929 20:47:45.680801 991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95476e3b89fe467ea3f151400dcc4f76" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86"
I20260929 20:47:45.682993 1390 tablet_bootstrap.cc:492] T 95476e3b89fe467ea3f151400dcc4f76 P d1f79a0bcbdb42fe89e7812efeae1a86: No bootstrap required, opened a new log
I20260929 20:47:45.683214 1390 ts_tablet_manager.cc:1397] T 95476e3b89fe467ea3f151400dcc4f76 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent bootstrapping tablet: real 0.122s user 0.012s sys 0.040s
I20260929 20:47:45.684002 1390 raft_consensus.cc:348] T 95476e3b89fe467ea3f151400dcc4f76 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:45.684265 1390 raft_consensus.cc:374] T 95476e3b89fe467ea3f151400dcc4f76 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.684347 1390 raft_consensus.cc:729] T 95476e3b89fe467ea3f151400dcc4f76 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.672900 1412 consensus_queue.cc:237] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
W20260929 20:47:45.684742 1069 leader_election.cc:341] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): Illegal state: must be running to vote when last-logged opid is not known
I20260929 20:47:45.685240 1391 tablet_bootstrap.cc:492] T c4a57da4e5c24803b7aa3f61c7191e38 P 1cb32e9690a546149d11339d371c832f: No bootstrap required, opened a new log
I20260929 20:47:45.685050 1069 leader_election.cc:302] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298; no voters: d1f79a0bcbdb42fe89e7812efeae1a86
I20260929 20:47:45.685456 1391 ts_tablet_manager.cc:1397] T c4a57da4e5c24803b7aa3f61c7191e38 P 1cb32e9690a546149d11339d371c832f: Time spent bootstrapping tablet: real 0.035s user 0.011s sys 0.004s
I20260929 20:47:45.685792 1397 raft_consensus.cc:2781] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 20:47:45.685997 1397 raft_consensus.cc:686] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [term 1 LEADER]: Becoming Leader. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Running, Role: LEADER
I20260929 20:47:45.686264 1391 raft_consensus.cc:348] T c4a57da4e5c24803b7aa3f61c7191e38 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.686512 1391 raft_consensus.cc:374] T c4a57da4e5c24803b7aa3f61c7191e38 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.686604 1391 raft_consensus.cc:729] T c4a57da4e5c24803b7aa3f61c7191e38 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.686514 1390 consensus_queue.cc:260] T 95476e3b89fe467ea3f151400dcc4f76 P d1f79a0bcbdb42fe89e7812efeae1a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:45.686856 1391 consensus_queue.cc:260] T c4a57da4e5c24803b7aa3f61c7191e38 P 1cb32e9690a546149d11339d371c832f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.687137 1390 ts_tablet_manager.cc:1428] T 95476e3b89fe467ea3f151400dcc4f76 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260929 20:47:45.687332 1391 ts_tablet_manager.cc:1428] T c4a57da4e5c24803b7aa3f61c7191e38 P 1cb32e9690a546149d11339d371c832f: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260929 20:47:45.687426 1390 tablet_bootstrap.cc:492] T c4a57da4e5c24803b7aa3f61c7191e38 P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap starting.
I20260929 20:47:45.687618 1391 tablet_bootstrap.cc:492] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f: Bootstrap starting.
I20260929 20:47:45.689189 1390 tablet_bootstrap.cc:654] T c4a57da4e5c24803b7aa3f61c7191e38 P d1f79a0bcbdb42fe89e7812efeae1a86: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.690749 1391 tablet_bootstrap.cc:654] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.699186 1397 consensus_queue.cc:237] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:45.702086 1390 tablet_bootstrap.cc:492] T c4a57da4e5c24803b7aa3f61c7191e38 P d1f79a0bcbdb42fe89e7812efeae1a86: No bootstrap required, opened a new log
I20260929 20:47:45.702302 1390 ts_tablet_manager.cc:1397] T c4a57da4e5c24803b7aa3f61c7191e38 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260929 20:47:45.703186 1390 raft_consensus.cc:348] T c4a57da4e5c24803b7aa3f61c7191e38 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.703470 1390 raft_consensus.cc:374] T c4a57da4e5c24803b7aa3f61c7191e38 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.703563 1390 raft_consensus.cc:729] T c4a57da4e5c24803b7aa3f61c7191e38 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.704133 1390 consensus_queue.cc:260] T c4a57da4e5c24803b7aa3f61c7191e38 P d1f79a0bcbdb42fe89e7812efeae1a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.705622 1390 ts_tablet_manager.cc:1428] T c4a57da4e5c24803b7aa3f61c7191e38 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260929 20:47:45.705919 1390 tablet_bootstrap.cc:492] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap starting.
I20260929 20:47:45.707619 1390 tablet_bootstrap.cc:654] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.709707 1388 tablet_bootstrap.cc:492] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298: No bootstrap required, opened a new log
I20260929 20:47:45.709875 1388 ts_tablet_manager.cc:1397] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298: Time spent bootstrapping tablet: real 0.143s user 0.024s sys 0.039s
I20260929 20:47:45.710628 1388 raft_consensus.cc:348] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.710918 1388 raft_consensus.cc:374] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.711006 1388 raft_consensus.cc:729] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.711283 1388 consensus_queue.cc:260] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.713270 1388 ts_tablet_manager.cc:1428] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260929 20:47:45.714493 1388 tablet_bootstrap.cc:492] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298: Bootstrap starting.
I20260929 20:47:45.716241 1388 tablet_bootstrap.cc:654] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.717857 1390 tablet_bootstrap.cc:492] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86: No bootstrap required, opened a new log
I20260929 20:47:45.718026 1390 ts_tablet_manager.cc:1397] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20260929 20:47:45.718806 1390 raft_consensus.cc:348] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.719080 1390 raft_consensus.cc:374] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.719161 1390 raft_consensus.cc:729] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.719456 1390 consensus_queue.cc:260] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.720856 1390 ts_tablet_manager.cc:1428] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 20:47:45.721150 1390 tablet_bootstrap.cc:492] T 9201e11b19f84f0db4e2b132a1b60ff8 P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap starting.
I20260929 20:47:45.721997 1391 tablet_bootstrap.cc:492] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f: No bootstrap required, opened a new log
I20260929 20:47:45.722149 1391 ts_tablet_manager.cc:1397] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f: Time spent bootstrapping tablet: real 0.035s user 0.010s sys 0.006s
I20260929 20:47:45.722887 1391 raft_consensus.cc:348] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.723088 1390 tablet_bootstrap.cc:654] T 9201e11b19f84f0db4e2b132a1b60ff8 P d1f79a0bcbdb42fe89e7812efeae1a86: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.723106 1391 raft_consensus.cc:374] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.723182 1391 raft_consensus.cc:729] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.723455 1391 consensus_queue.cc:260] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.725970 1391 ts_tablet_manager.cc:1428] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260929 20:47:45.727082 1391 tablet_bootstrap.cc:492] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f: Bootstrap starting.
I20260929 20:47:45.728782 1391 tablet_bootstrap.cc:654] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.729435 1388 tablet_bootstrap.cc:492] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298: No bootstrap required, opened a new log
I20260929 20:47:45.729600 1388 ts_tablet_manager.cc:1397] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.003s
I20260929 20:47:45.730643 1388 raft_consensus.cc:348] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.730903 1388 raft_consensus.cc:374] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.731225 1388 raft_consensus.cc:729] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.731521 1388 consensus_queue.cc:260] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.732105 1388 ts_tablet_manager.cc:1428] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260929 20:47:45.732443 1388 tablet_bootstrap.cc:492] T 9201e11b19f84f0db4e2b132a1b60ff8 P cac815b9c12e40c6b63d4af82248f298: Bootstrap starting.
I20260929 20:47:45.734383 1388 tablet_bootstrap.cc:654] T 9201e11b19f84f0db4e2b132a1b60ff8 P cac815b9c12e40c6b63d4af82248f298: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.736721 1388 tablet_bootstrap.cc:492] T 9201e11b19f84f0db4e2b132a1b60ff8 P cac815b9c12e40c6b63d4af82248f298: No bootstrap required, opened a new log
I20260929 20:47:45.736874 1388 ts_tablet_manager.cc:1397] T 9201e11b19f84f0db4e2b132a1b60ff8 P cac815b9c12e40c6b63d4af82248f298: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260929 20:47:45.737643 1388 raft_consensus.cc:348] T 9201e11b19f84f0db4e2b132a1b60ff8 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.737901 1388 raft_consensus.cc:374] T 9201e11b19f84f0db4e2b132a1b60ff8 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.738003 1388 raft_consensus.cc:729] T 9201e11b19f84f0db4e2b132a1b60ff8 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.738246 1388 consensus_queue.cc:260] T 9201e11b19f84f0db4e2b132a1b60ff8 P cac815b9c12e40c6b63d4af82248f298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.739025 1388 ts_tablet_manager.cc:1428] T 9201e11b19f84f0db4e2b132a1b60ff8 P cac815b9c12e40c6b63d4af82248f298: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260929 20:47:45.739740 1044 tablet.cc:2369] T 1982dc006d3c45bcb4f843e7067929ee P d1f79a0bcbdb42fe89e7812efeae1a86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 20:47:45.740425 1390 tablet_bootstrap.cc:492] T 9201e11b19f84f0db4e2b132a1b60ff8 P d1f79a0bcbdb42fe89e7812efeae1a86: No bootstrap required, opened a new log
I20260929 20:47:45.740617 1390 ts_tablet_manager.cc:1397] T 9201e11b19f84f0db4e2b132a1b60ff8 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.002s
I20260929 20:47:45.741451 1390 raft_consensus.cc:348] T 9201e11b19f84f0db4e2b132a1b60ff8 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.741694 1390 raft_consensus.cc:374] T 9201e11b19f84f0db4e2b132a1b60ff8 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.741823 1390 raft_consensus.cc:729] T 9201e11b19f84f0db4e2b132a1b60ff8 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.742131 1390 consensus_queue.cc:260] T 9201e11b19f84f0db4e2b132a1b60ff8 P d1f79a0bcbdb42fe89e7812efeae1a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.742811 1390 ts_tablet_manager.cc:1428] T 9201e11b19f84f0db4e2b132a1b60ff8 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 20:47:45.743100 1390 tablet_bootstrap.cc:492] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap starting.
I20260929 20:47:45.744093 1391 tablet_bootstrap.cc:492] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f: No bootstrap required, opened a new log
I20260929 20:47:45.744269 1391 ts_tablet_manager.cc:1397] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.004s
I20260929 20:47:45.744874 1390 tablet_bootstrap.cc:654] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.745115 1391 raft_consensus.cc:348] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.745362 1391 raft_consensus.cc:374] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.745476 1391 raft_consensus.cc:729] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.745764 1391 consensus_queue.cc:260] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.748880 1391 ts_tablet_manager.cc:1428] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260929 20:47:45.754434 1390 tablet_bootstrap.cc:492] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86: No bootstrap required, opened a new log
I20260929 20:47:45.754653 1390 ts_tablet_manager.cc:1397] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260929 20:47:45.755650 1390 raft_consensus.cc:348] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.755918 1390 raft_consensus.cc:374] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.756006 1390 raft_consensus.cc:729] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.756281 1390 consensus_queue.cc:260] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.758823 1390 ts_tablet_manager.cc:1428] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
20:47:45.804 [HMS - ERROR - pool-6-thread-2] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_4cf3799777554d47b5dbb898a6d368b9 already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy25.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
W20260929 20:47:45.807293 656 catalog_manager.cc:2144] Already present: failed to create HMS catalog entry for table [id=3b5f2721f164493491ba371c0d990976]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_4cf3799777554d47b5dbb898a6d368b9 already exists)
I20260929 20:47:45.814733 970 tablet_service.cc:1514] Processing DeleteTablet for tablet 1982dc006d3c45bcb4f843e7067929ee with delete_type TABLET_DATA_DELETED (replaced by tablet e0bb85fb9e3d49be88b7fa9dccec6cda) from {username='slave'} at 127.0.0.1:47676
I20260929 20:47:45.815375 971 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac5f47c77a842aca9f996f49f34daf0 with delete_type TABLET_DATA_DELETED (replaced by tablet c4a57da4e5c24803b7aa3f61c7191e38) from {username='slave'} at 127.0.0.1:47676
I20260929 20:47:45.816448 1392 tablet_replica.cc:330] T 1982dc006d3c45bcb4f843e7067929ee P d1f79a0bcbdb42fe89e7812efeae1a86: stopping tablet replica
I20260929 20:47:45.816915 1392 raft_consensus.cc:2229] T 1982dc006d3c45bcb4f843e7067929ee P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260929 20:47:45.817314 1392 raft_consensus.cc:2258] T 1982dc006d3c45bcb4f843e7067929ee P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260929 20:47:45.817713 1116 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac5f47c77a842aca9f996f49f34daf0 with delete_type TABLET_DATA_DELETED (replaced by tablet c4a57da4e5c24803b7aa3f61c7191e38) from {username='slave'} at 127.0.0.1:35688
I20260929 20:47:45.817965 1115 tablet_service.cc:1514] Processing DeleteTablet for tablet 1982dc006d3c45bcb4f843e7067929ee with delete_type TABLET_DATA_DELETED (replaced by tablet e0bb85fb9e3d49be88b7fa9dccec6cda) from {username='slave'} at 127.0.0.1:35688
I20260929 20:47:45.818327 1394 tablet_replica.cc:330] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298: stopping tablet replica
I20260929 20:47:45.818723 1394 raft_consensus.cc:2229] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [term 1 LEADER]: Raft consensus shutting down.
I20260929 20:47:45.819208 1394 pending_rounds.cc:62] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298: Trying to abort 1 pending ops.
I20260929 20:47:45.819320 1394 pending_rounds.cc:68] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334768089848135680 op_type: NO_OP noop_request { }
I20260929 20:47:45.819557 1394 raft_consensus.cc:2866] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260929 20:47:45.819661 1394 raft_consensus.cc:2258] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 20:47:45.822667 1392 ts_tablet_manager.cc:1905] T 1982dc006d3c45bcb4f843e7067929ee P d1f79a0bcbdb42fe89e7812efeae1a86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 20:47:45.825789 1412 raft_consensus.cc:482] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:45.825975 1412 raft_consensus.cc:504] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.826689 1412 leader_election.cc:288] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:45.827523 991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a57da4e5c24803b7aa3f61c7191e38" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" is_pre_election: true
I20260929 20:47:45.827535 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a57da4e5c24803b7aa3f61c7191e38" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f" is_pre_election: true
I20260929 20:47:45.827766 991 raft_consensus.cc:2454] T c4a57da4e5c24803b7aa3f61c7191e38 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cac815b9c12e40c6b63d4af82248f298 in term 0.
I20260929 20:47:45.827786 1278 raft_consensus.cc:2454] T c4a57da4e5c24803b7aa3f61c7191e38 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cac815b9c12e40c6b63d4af82248f298 in term 0.
I20260929 20:47:45.828262 1069 leader_election.cc:302] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cac815b9c12e40c6b63d4af82248f298, d1f79a0bcbdb42fe89e7812efeae1a86; no voters:
I20260929 20:47:45.828776 1412 raft_consensus.cc:2781] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 20:47:45.828893 1412 raft_consensus.cc:482] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:47:45.829001 1412 raft_consensus.cc:3037] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:45.830433 1394 ts_tablet_manager.cc:1905] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 20:47:45.831869 1412 raft_consensus.cc:504] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.833096 991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a57da4e5c24803b7aa3f61c7191e38" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86"
I20260929 20:47:45.833357 991 raft_consensus.cc:3037] T c4a57da4e5c24803b7aa3f61c7191e38 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:45.836055 991 raft_consensus.cc:2454] T c4a57da4e5c24803b7aa3f61c7191e38 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cac815b9c12e40c6b63d4af82248f298 in term 1.
I20260929 20:47:45.836640 1069 leader_election.cc:302] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cac815b9c12e40c6b63d4af82248f298, d1f79a0bcbdb42fe89e7812efeae1a86; no voters:
I20260929 20:47:45.836957 1397 raft_consensus.cc:2781] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 20:47:45.837133 1397 raft_consensus.cc:686] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [term 1 LEADER]: Becoming Leader. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Running, Role: LEADER
I20260929 20:47:45.837427 1397 consensus_queue.cc:237] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.840101 1412 leader_election.cc:288] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 election: Requested vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:45.840078 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a57da4e5c24803b7aa3f61c7191e38" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f"
I20260929 20:47:45.840284 1278 raft_consensus.cc:3037] T c4a57da4e5c24803b7aa3f61c7191e38 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:45.842906 1278 raft_consensus.cc:2454] T c4a57da4e5c24803b7aa3f61c7191e38 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cac815b9c12e40c6b63d4af82248f298 in term 1.
I20260929 20:47:45.845140 1392 ts_tablet_manager.cc:1918] T 1982dc006d3c45bcb4f843e7067929ee P d1f79a0bcbdb42fe89e7812efeae1a86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260929 20:47:45.847031 1258 tablet_service.cc:1514] Processing DeleteTablet for tablet 1982dc006d3c45bcb4f843e7067929ee with delete_type TABLET_DATA_DELETED (replaced by tablet e0bb85fb9e3d49be88b7fa9dccec6cda) from {username='slave'} at 127.0.0.1:54804
I20260929 20:47:45.847327 1393 tablet_replica.cc:330] T 1982dc006d3c45bcb4f843e7067929ee P 1cb32e9690a546149d11339d371c832f: stopping tablet replica
I20260929 20:47:45.847791 1393 raft_consensus.cc:2229] T 1982dc006d3c45bcb4f843e7067929ee P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260929 20:47:45.848187 1393 raft_consensus.cc:2258] T 1982dc006d3c45bcb4f843e7067929ee P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260929 20:47:45.849997 1393 ts_tablet_manager.cc:1905] T 1982dc006d3c45bcb4f843e7067929ee P 1cb32e9690a546149d11339d371c832f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 20:47:45.857578 1394 ts_tablet_manager.cc:1918] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260929 20:47:45.857870 1393 ts_tablet_manager.cc:1918] T 1982dc006d3c45bcb4f843e7067929ee P 1cb32e9690a546149d11339d371c832f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260929 20:47:45.858098 1393 log.cc:1196] T 1982dc006d3c45bcb4f843e7067929ee P 1cb32e9690a546149d11339d371c832f: Deleting WAL directory at /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/wal/wals/1982dc006d3c45bcb4f843e7067929ee
I20260929 20:47:45.858554 1393 ts_tablet_manager.cc:1939] T 1982dc006d3c45bcb4f843e7067929ee P 1cb32e9690a546149d11339d371c832f: Deleting consensus metadata
I20260929 20:47:45.859797 1392 log.cc:1196] T 1982dc006d3c45bcb4f843e7067929ee P d1f79a0bcbdb42fe89e7812efeae1a86: Deleting WAL directory at /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/wal/wals/1982dc006d3c45bcb4f843e7067929ee
I20260929 20:47:45.860793 1258 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac5f47c77a842aca9f996f49f34daf0 with delete_type TABLET_DATA_DELETED (replaced by tablet c4a57da4e5c24803b7aa3f61c7191e38) from {username='slave'} at 127.0.0.1:54804
I20260929 20:47:45.861217 1394 log.cc:1196] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298: Deleting WAL directory at /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/wal/wals/4ac5f47c77a842aca9f996f49f34daf0
I20260929 20:47:45.863781 1392 ts_tablet_manager.cc:1939] T 1982dc006d3c45bcb4f843e7067929ee P d1f79a0bcbdb42fe89e7812efeae1a86: Deleting consensus metadata
I20260929 20:47:45.864356 1394 ts_tablet_manager.cc:1939] T 4ac5f47c77a842aca9f996f49f34daf0 P cac815b9c12e40c6b63d4af82248f298: Deleting consensus metadata
I20260929 20:47:45.866236 656 catalog_manager.cc:5495] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 reported cstate change: term changed from 0 to 1, leader changed from <none> to cac815b9c12e40c6b63d4af82248f298 (127.0.97.2). New cstate: current_term: 1 leader_uuid: "cac815b9c12e40c6b63d4af82248f298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } health_report { overall_health: UNKNOWN } } }
I20260929 20:47:45.871838 1393 tablet_replica.cc:330] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f: stopping tablet replica
I20260929 20:47:45.872144 1393 raft_consensus.cc:2229] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 20:47:45.873392 1393 raft_consensus.cc:2258] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 20:47:45.873757 1392 tablet_replica.cc:330] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86: stopping tablet replica
I20260929 20:47:45.874015 1394 tablet_replica.cc:330] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298: stopping tablet replica
I20260929 20:47:45.874269 1392 raft_consensus.cc:2229] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 20:47:45.874502 1394 raft_consensus.cc:2229] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260929 20:47:45.874851 1392 raft_consensus.cc:2258] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 20:47:45.875010 1394 raft_consensus.cc:2258] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260929 20:47:45.868120 645 catalog_manager.cc:4841] TS 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): tablet 1982dc006d3c45bcb4f843e7067929ee (table default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]) successfully deleted
I20260929 20:47:45.875351 645 catalog_manager.cc:4841] TS d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): tablet 1982dc006d3c45bcb4f843e7067929ee (table default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]) successfully deleted
I20260929 20:47:45.870260 646 catalog_manager.cc:4841] TS cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): tablet 4ac5f47c77a842aca9f996f49f34daf0 (table default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f]) successfully deleted
I20260929 20:47:45.873478 660 catalog_manager.cc:2148] created HMS catalog entry for table [id=dd1c5cbd5c6c40fc9fb5cc1dbe7026b0]
I20260929 20:47:45.876714 1393 ts_tablet_manager.cc:1905] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 20:47:45.877421 1392 ts_tablet_manager.cc:1905] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 20:47:45.884580 1393 ts_tablet_manager.cc:1918] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260929 20:47:45.884747 1393 log.cc:1196] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f: Deleting WAL directory at /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/wal/wals/4ac5f47c77a842aca9f996f49f34daf0
I20260929 20:47:45.885089 1393 ts_tablet_manager.cc:1939] T 4ac5f47c77a842aca9f996f49f34daf0 P 1cb32e9690a546149d11339d371c832f: Deleting consensus metadata
I20260929 20:47:45.885273 1394 ts_tablet_manager.cc:1905] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 20:47:45.886247 645 catalog_manager.cc:4841] TS 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): tablet 4ac5f47c77a842aca9f996f49f34daf0 (table default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f]) successfully deleted
I20260929 20:47:45.911902 1397 raft_consensus.cc:482] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:45.912091 1397 raft_consensus.cc:504] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.912755 1397 leader_election.cc:288] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:45.914729 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0bb85fb9e3d49be88b7fa9dccec6cda" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f" is_pre_election: true
I20260929 20:47:45.915030 1278 raft_consensus.cc:2454] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cac815b9c12e40c6b63d4af82248f298 in term 0.
I20260929 20:47:45.915763 1400 raft_consensus.cc:482] T 95476e3b89fe467ea3f151400dcc4f76 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:45.915966 1400 raft_consensus.cc:504] T 95476e3b89fe467ea3f151400dcc4f76 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:45.916705 991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0bb85fb9e3d49be88b7fa9dccec6cda" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" is_pre_election: true
I20260929 20:47:45.916953 991 raft_consensus.cc:2454] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cac815b9c12e40c6b63d4af82248f298 in term 0.
I20260929 20:47:45.918066 1069 leader_election.cc:302] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298; no voters:
I20260929 20:47:45.918380 1397 raft_consensus.cc:2781] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 20:47:45.918491 1397 raft_consensus.cc:482] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:47:45.918583 1397 raft_consensus.cc:3037] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:45.922995 1400 leader_election.cc:288] T 95476e3b89fe467ea3f151400dcc4f76 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:45.925357 1258 tablet_service.cc:1467] Processing CreateTablet for tablet 6820644bb2a745679d959d8ac6854014 (DEFAULT_TABLE table=default.table_f02a75c1e46d472d90334b4340d99e7b [id=dd1c5cbd5c6c40fc9fb5cc1dbe7026b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.925982 1258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6820644bb2a745679d959d8ac6854014. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.928646 1257 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac5f47c77a842aca9f996f49f34daf0 with delete_type TABLET_DATA_DELETED (replaced by tablet c4a57da4e5c24803b7aa3f61c7191e38) from {username='slave'} at 127.0.0.1:54804
W20260929 20:47:45.929970 645 catalog_manager.cc:4808] TS 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): delete failed for tablet 4ac5f47c77a842aca9f996f49f34daf0 because the tablet was not found. No further retry: Not found: Tablet not found: 4ac5f47c77a842aca9f996f49f34daf0
I20260929 20:47:45.931524 660 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35484:
name: "default.Table_a988b1827f9e482086b9b9ad0663c2b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:45.932427 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95476e3b89fe467ea3f151400dcc4f76" candidate_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f" is_pre_election: true
I20260929 20:47:45.932812 1278 raft_consensus.cc:2379] T 95476e3b89fe467ea3f151400dcc4f76 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1f79a0bcbdb42fe89e7812efeae1a86 in current term 1: Already voted for candidate cac815b9c12e40c6b63d4af82248f298 in this term.
I20260929 20:47:45.934907 1397 raft_consensus.cc:504] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.935993 1397 leader_election.cc:288] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 election: Requested vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:45.936264 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0bb85fb9e3d49be88b7fa9dccec6cda" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f"
I20260929 20:47:45.936484 1278 raft_consensus.cc:3037] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:45.939381 1278 raft_consensus.cc:2454] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cac815b9c12e40c6b63d4af82248f298 in term 1.
I20260929 20:47:45.939874 1069 leader_election.cc:302] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298; no voters:
I20260929 20:47:45.940340 1397 raft_consensus.cc:2781] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 20:47:45.940549 1397 raft_consensus.cc:686] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [term 1 LEADER]: Becoming Leader. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Running, Role: LEADER
I20260929 20:47:45.940840 1397 consensus_queue.cc:237] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:45.944322 1391 tablet_bootstrap.cc:492] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f: Bootstrap starting.
I20260929 20:47:45.945991 1391 tablet_bootstrap.cc:654] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.934475 655 catalog_manager.cc:2148] created HMS catalog entry for table [id=3842db9126434978a3631262acd85d32]
I20260929 20:47:45.953369 655 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35484 (request call id 8) took 1024 ms. Trace:
I20260929 20:47:45.953570 655 rpcz_store.cc:276] 0929 20:47:44.928487 (+ 0us) service_pool.cc:170] Inserting onto call queue
0929 20:47:44.931223 (+ 2736us) service_pool.cc:227] Handling call
0929 20:47:45.240588 (+309365us) catalog_manager.cc:2075] Acquired catalog manager lock
0929 20:47:45.241048 (+ 460us) catalog_manager.cc:2121] Created new table and tablet info
0929 20:47:45.934429 (+693381us) catalog_manager.cc:2147] Created new table in HMS catalog
0929 20:47:45.953051 (+ 18622us) catalog_manager.cc:2181] Wrote table and tablets to system table
0929 20:47:45.953210 (+ 159us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0929 20:47:45.953318 (+ 108us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0929 20:47:45.947426 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0929 20:47:45.947558 (+ 132us) write_op.cc:432] Acquiring schema lock in shared mode
0929 20:47:45.947563 (+ 5us) write_op.cc:435] Acquired schema lock
0929 20:47:45.947565 (+ 2us) tablet.cc:660] Decoding operations
0929 20:47:45.947687 (+ 122us) write_op.cc:620] Acquiring the partition lock for write op
0929 20:47:45.947699 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0929 20:47:45.947702 (+ 3us) tablet.cc:683] Acquiring locks for 2 operations
0929 20:47:45.947778 (+ 76us) tablet.cc:699] Row locks acquired
0929 20:47:45.947779 (+ 1us) write_op.cc:260] PREPARE: finished
0929 20:47:45.947812 (+ 33us) write_op.cc:270] Start()
0929 20:47:45.947841 (+ 29us) write_op.cc:276] Timestamp: P: 1790714865947736 usec, L: 0
0929 20:47:45.947843 (+ 2us) op_driver.cc:331] REPLICATION: starting
0929 20:47:45.948045 (+ 202us) log.cc:842] Serialized 432 byte log entry
0929 20:47:45.952139 (+ 4094us) op_driver.cc:444] REPLICATION: finished
0929 20:47:45.952333 (+ 194us) write_op.cc:301] APPLY: starting
0929 20:47:45.952364 (+ 31us) tablet.cc:1365] starting BulkCheckPresence
0929 20:47:45.952421 (+ 57us) tablet.cc:1368] finished BulkCheckPresence
0929 20:47:45.952467 (+ 46us) tablet.cc:1370] starting ApplyRowOperation cycle
0929 20:47:45.952634 (+ 167us) tablet.cc:1381] finished ApplyRowOperation cycle
0929 20:47:45.952679 (+ 45us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0929 20:47:45.952688 (+ 9us) write_op.cc:312] APPLY: finished
0929 20:47:45.952786 (+ 98us) log.cc:842] Serialized 28 byte log entry
0929 20:47:45.952866 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0929 20:47:45.952888 (+ 22us) write_op.cc:454] Released schema lock
0929 20:47:45.952895 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0929 20:47:45.952951 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0929 20:47:45.952957 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":632160,"Hive Metastore.run_cpu_time_us":286,"Hive Metastore.run_wall_time_us":60866,"child_traces":[["op",{"apply.queue_time_us":134,"apply.run_cpu_time_us":677,"apply.run_wall_time_us":677,"num_ops":2,"prepare.queue_time_us":46,"prepare.run_cpu_time_us":759,"prepare.run_wall_time_us":759,"raft.queue_time_us":188,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":312,"replication_time_us":4264}]]}
I20260929 20:47:45.955485 655 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35484:
name: "default.Table_e1a3cd10e94d42c598e9c071db2144d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:45.961918 656 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35484:
name: "default.Table_dae8faea7e2b44909906d91d85898732"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:45.968422 1258 tablet_service.cc:1467] Processing CreateTablet for tablet 2639596901a04bdb8bdbf8ccae3d1a50 (DEFAULT_TABLE table=default.table_5b5c251fd82449869826c01b42a62f31 [id=3842db9126434978a3631262acd85d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.969847 1258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2639596901a04bdb8bdbf8ccae3d1a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.974073 1394 ts_tablet_manager.cc:1918] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260929 20:47:45.977406 991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0bb85fb9e3d49be88b7fa9dccec6cda" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86"
I20260929 20:47:45.977670 991 raft_consensus.cc:3037] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:45.980928 991 raft_consensus.cc:2454] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cac815b9c12e40c6b63d4af82248f298 in term 1.
I20260929 20:47:45.981030 1392 ts_tablet_manager.cc:1918] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260929 20:47:45.983405 1392 log.cc:1196] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86: Deleting WAL directory at /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/wal/wals/4ac5f47c77a842aca9f996f49f34daf0
I20260929 20:47:45.984515 971 tablet_service.cc:1467] Processing CreateTablet for tablet 6820644bb2a745679d959d8ac6854014 (DEFAULT_TABLE table=default.table_f02a75c1e46d472d90334b4340d99e7b [id=dd1c5cbd5c6c40fc9fb5cc1dbe7026b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:45.984988 971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6820644bb2a745679d959d8ac6854014. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:45.986660 1391 tablet_bootstrap.cc:492] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f: No bootstrap required, opened a new log
I20260929 20:47:45.986800 1391 ts_tablet_manager.cc:1397] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f: Time spent bootstrapping tablet: real 0.043s user 0.005s sys 0.006s
I20260929 20:47:45.987574 1391 raft_consensus.cc:348] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:45.987784 1391 raft_consensus.cc:374] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:45.987856 1391 raft_consensus.cc:729] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Initialized, Role: FOLLOWER
I20260929 20:47:45.988085 1391 consensus_queue.cc:260] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:45.991751 1391 ts_tablet_manager.cc:1428] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260929 20:47:45.992116 1390 tablet_bootstrap.cc:492] T 6820644bb2a745679d959d8ac6854014 P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap starting.
I20260929 20:47:45.993357 1391 tablet_bootstrap.cc:492] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f: Bootstrap starting.
I20260929 20:47:45.993909 1390 tablet_bootstrap.cc:654] T 6820644bb2a745679d959d8ac6854014 P d1f79a0bcbdb42fe89e7812efeae1a86: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.995147 1391 tablet_bootstrap.cc:654] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:45.996690 1392 ts_tablet_manager.cc:1939] T 4ac5f47c77a842aca9f996f49f34daf0 P d1f79a0bcbdb42fe89e7812efeae1a86: Deleting consensus metadata
I20260929 20:47:45.999135 1391 tablet_bootstrap.cc:492] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f: No bootstrap required, opened a new log
I20260929 20:47:45.999332 1391 ts_tablet_manager.cc:1397] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20260929 20:47:46.000130 1391 raft_consensus.cc:348] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:46.000550 1391 raft_consensus.cc:374] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:46.000705 1391 raft_consensus.cc:729] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Initialized, Role: FOLLOWER
I20260929 20:47:46.001128 1391 consensus_queue.cc:260] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:46.003813 971 tablet_service.cc:1514] Processing DeleteTablet for tablet 1982dc006d3c45bcb4f843e7067929ee with delete_type TABLET_DATA_DELETED (replaced by tablet e0bb85fb9e3d49be88b7fa9dccec6cda) from {username='slave'} at 127.0.0.1:47676
I20260929 20:47:46.003993 1391 ts_tablet_manager.cc:1428] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260929 20:47:46.004917 971 tablet_service.cc:1467] Processing CreateTablet for tablet 2639596901a04bdb8bdbf8ccae3d1a50 (DEFAULT_TABLE table=default.table_5b5c251fd82449869826c01b42a62f31 [id=3842db9126434978a3631262acd85d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.005403 971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2639596901a04bdb8bdbf8ccae3d1a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:46.009636 645 catalog_manager.cc:4841] TS d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): tablet 4ac5f47c77a842aca9f996f49f34daf0 (table default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f]) successfully deleted
W20260929 20:47:46.010121 645 catalog_manager.cc:4808] TS d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): delete failed for tablet 1982dc006d3c45bcb4f843e7067929ee because the tablet was not found. No further retry: Not found: Tablet not found: 1982dc006d3c45bcb4f843e7067929ee
I20260929 20:47:46.010509 1400 raft_consensus.cc:482] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:46.010661 1400 raft_consensus.cc:504] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:46.011258 1400 leader_election.cc:288] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:46.012046 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53c949bb99444778a9f15a781596e986" candidate_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f" is_pre_election: true
I20260929 20:47:46.012291 1278 raft_consensus.cc:2454] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1f79a0bcbdb42fe89e7812efeae1a86 in term 0.
I20260929 20:47:46.012785 924 leader_election.cc:302] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, d1f79a0bcbdb42fe89e7812efeae1a86; no voters:
I20260929 20:47:46.013326 1400 raft_consensus.cc:2781] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 20:47:46.013420 1400 raft_consensus.cc:482] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:47:46.013528 1400 raft_consensus.cc:3037] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.016278 1390 tablet_bootstrap.cc:492] T 6820644bb2a745679d959d8ac6854014 P d1f79a0bcbdb42fe89e7812efeae1a86: No bootstrap required, opened a new log
I20260929 20:47:46.016548 1390 ts_tablet_manager.cc:1397] T 6820644bb2a745679d959d8ac6854014 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.004s
I20260929 20:47:46.016552 1400 raft_consensus.cc:504] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:46.017491 1400 leader_election.cc:288] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 1 election: Requested vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:46.017539 1390 raft_consensus.cc:348] T 6820644bb2a745679d959d8ac6854014 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:46.017899 1390 raft_consensus.cc:374] T 6820644bb2a745679d959d8ac6854014 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:46.017911 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53c949bb99444778a9f15a781596e986" candidate_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f"
I20260929 20:47:46.018028 1390 raft_consensus.cc:729] T 6820644bb2a745679d959d8ac6854014 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Initialized, Role: FOLLOWER
I20260929 20:47:46.018123 1278 raft_consensus.cc:3037] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.018378 1390 consensus_queue.cc:260] T 6820644bb2a745679d959d8ac6854014 P d1f79a0bcbdb42fe89e7812efeae1a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:46.019004 1390 ts_tablet_manager.cc:1428] T 6820644bb2a745679d959d8ac6854014 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 20:47:46.022553 1278 raft_consensus.cc:2454] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1f79a0bcbdb42fe89e7812efeae1a86 in term 1.
I20260929 20:47:46.023087 924 leader_election.cc:302] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, d1f79a0bcbdb42fe89e7812efeae1a86; no voters:
I20260929 20:47:46.023420 1400 raft_consensus.cc:2781] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 20:47:46.001297 1115 tablet_service.cc:1467] Processing CreateTablet for tablet 6820644bb2a745679d959d8ac6854014 (DEFAULT_TABLE table=default.table_f02a75c1e46d472d90334b4340d99e7b [id=dd1c5cbd5c6c40fc9fb5cc1dbe7026b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.024744 1394 log.cc:1196] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298: Deleting WAL directory at /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/wal/wals/1982dc006d3c45bcb4f843e7067929ee
I20260929 20:47:46.025075 1115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6820644bb2a745679d959d8ac6854014. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:46.025358 1394 ts_tablet_manager.cc:1939] T 1982dc006d3c45bcb4f843e7067929ee P cac815b9c12e40c6b63d4af82248f298: Deleting consensus metadata
I20260929 20:47:46.002295 1116 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac5f47c77a842aca9f996f49f34daf0 with delete_type TABLET_DATA_DELETED (replaced by tablet c4a57da4e5c24803b7aa3f61c7191e38) from {username='slave'} at 127.0.0.1:35688
I20260929 20:47:46.002772 1114 tablet_service.cc:1467] Processing CreateTablet for tablet 2639596901a04bdb8bdbf8ccae3d1a50 (DEFAULT_TABLE table=default.table_5b5c251fd82449869826c01b42a62f31 [id=3842db9126434978a3631262acd85d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.028957 1114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2639596901a04bdb8bdbf8ccae3d1a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:46.021667 1136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95476e3b89fe467ea3f151400dcc4f76" candidate_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298" is_pre_election: true
I20260929 20:47:46.024748 1134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53c949bb99444778a9f15a781596e986" candidate_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298"
I20260929 20:47:46.033435 1134 raft_consensus.cc:3037] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.035357 1390 tablet_bootstrap.cc:492] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap starting.
I20260929 20:47:46.036130 1134 raft_consensus.cc:2454] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1f79a0bcbdb42fe89e7812efeae1a86 in term 1.
I20260929 20:47:46.024135 1135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53c949bb99444778a9f15a781596e986" candidate_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298" is_pre_election: true
I20260929 20:47:46.036441 1135 raft_consensus.cc:2362] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d1f79a0bcbdb42fe89e7812efeae1a86 in term 1. Re-sending same reply.
I20260929 20:47:46.037036 1390 tablet_bootstrap.cc:654] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:46.038516 1388 tablet_bootstrap.cc:492] T 6820644bb2a745679d959d8ac6854014 P cac815b9c12e40c6b63d4af82248f298: Bootstrap starting.
I20260929 20:47:46.039736 1390 tablet_bootstrap.cc:492] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86: No bootstrap required, opened a new log
I20260929 20:47:46.039916 1390 ts_tablet_manager.cc:1397] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260929 20:47:46.040686 1390 raft_consensus.cc:348] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:46.040921 1390 raft_consensus.cc:374] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:46.041003 1390 raft_consensus.cc:729] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Initialized, Role: FOLLOWER
I20260929 20:47:46.041157 646 catalog_manager.cc:4841] TS cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): tablet 1982dc006d3c45bcb4f843e7067929ee (table default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]) successfully deleted
I20260929 20:47:46.041252 1390 consensus_queue.cc:260] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:46.041697 925 leader_election.cc:302] T 95476e3b89fe467ea3f151400dcc4f76 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:47:46.048686 1388 tablet_bootstrap.cc:654] T 6820644bb2a745679d959d8ac6854014 P cac815b9c12e40c6b63d4af82248f298: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:46.048642 1434 raft_consensus.cc:3037] T 95476e3b89fe467ea3f151400dcc4f76 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.052057 1390 ts_tablet_manager.cc:1428] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent starting tablet: real 0.012s user 0.003s sys 0.004s
I20260929 20:47:46.055100 1434 raft_consensus.cc:2726] T 95476e3b89fe467ea3f151400dcc4f76 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260929 20:47:46.062189 646 catalog_manager.cc:4808] TS cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): delete failed for tablet 4ac5f47c77a842aca9f996f49f34daf0 because the tablet was not found. No further retry: Not found: Tablet not found: 4ac5f47c77a842aca9f996f49f34daf0
I20260929 20:47:46.062340 654 catalog_manager.cc:2148] created HMS catalog entry for table [id=b17facaee1924496b1beaf3d695c11ab]
I20260929 20:47:46.067906 1388 tablet_bootstrap.cc:492] T 6820644bb2a745679d959d8ac6854014 P cac815b9c12e40c6b63d4af82248f298: No bootstrap required, opened a new log
I20260929 20:47:46.068074 1388 ts_tablet_manager.cc:1397] T 6820644bb2a745679d959d8ac6854014 P cac815b9c12e40c6b63d4af82248f298: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.008s
I20260929 20:47:46.068907 1388 raft_consensus.cc:348] T 6820644bb2a745679d959d8ac6854014 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:46.069154 1388 raft_consensus.cc:374] T 6820644bb2a745679d959d8ac6854014 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:46.069244 1388 raft_consensus.cc:729] T 6820644bb2a745679d959d8ac6854014 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Initialized, Role: FOLLOWER
I20260929 20:47:46.069489 1388 consensus_queue.cc:260] T 6820644bb2a745679d959d8ac6854014 P cac815b9c12e40c6b63d4af82248f298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:46.071007 654 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35484 (request call id 9) took 1139 ms. Trace:
I20260929 20:47:46.071137 1388 ts_tablet_manager.cc:1428] T 6820644bb2a745679d959d8ac6854014 P cac815b9c12e40c6b63d4af82248f298: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 20:47:46.071139 654 rpcz_store.cc:276] 0929 20:47:44.931352 (+ 0us) service_pool.cc:170] Inserting onto call queue
0929 20:47:44.931611 (+ 259us) service_pool.cc:227] Handling call
0929 20:47:45.241491 (+309880us) catalog_manager.cc:2075] Acquired catalog manager lock
0929 20:47:45.241954 (+ 463us) catalog_manager.cc:2121] Created new table and tablet info
0929 20:47:46.062284 (+820330us) catalog_manager.cc:2147] Created new table in HMS catalog
0929 20:47:46.070724 (+ 8440us) catalog_manager.cc:2181] Wrote table and tablets to system table
0929 20:47:46.070855 (+ 131us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0929 20:47:46.070965 (+ 110us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0929 20:47:46.063207 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0929 20:47:46.063375 (+ 168us) write_op.cc:432] Acquiring schema lock in shared mode
0929 20:47:46.063384 (+ 9us) write_op.cc:435] Acquired schema lock
0929 20:47:46.063389 (+ 5us) tablet.cc:660] Decoding operations
0929 20:47:46.063508 (+ 119us) write_op.cc:620] Acquiring the partition lock for write op
0929 20:47:46.063525 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0929 20:47:46.063529 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
0929 20:47:46.063637 (+ 108us) tablet.cc:699] Row locks acquired
0929 20:47:46.063641 (+ 4us) write_op.cc:260] PREPARE: finished
0929 20:47:46.063693 (+ 52us) write_op.cc:270] Start()
0929 20:47:46.063737 (+ 44us) write_op.cc:276] Timestamp: P: 1790714866063656 usec, L: 0
0929 20:47:46.063743 (+ 6us) op_driver.cc:331] REPLICATION: starting
0929 20:47:46.063982 (+ 239us) log.cc:842] Serialized 432 byte log entry
0929 20:47:46.068838 (+ 4856us) op_driver.cc:444] REPLICATION: finished
0929 20:47:46.068993 (+ 155us) write_op.cc:301] APPLY: starting
0929 20:47:46.069009 (+ 16us) tablet.cc:1365] starting BulkCheckPresence
0929 20:47:46.069055 (+ 46us) tablet.cc:1368] finished BulkCheckPresence
0929 20:47:46.069089 (+ 34us) tablet.cc:1370] starting ApplyRowOperation cycle
0929 20:47:46.069208 (+ 119us) tablet.cc:1381] finished ApplyRowOperation cycle
0929 20:47:46.069251 (+ 43us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0929 20:47:46.069261 (+ 10us) write_op.cc:312] APPLY: finished
0929 20:47:46.069368 (+ 107us) log.cc:842] Serialized 28 byte log entry
0929 20:47:46.069433 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
0929 20:47:46.069455 (+ 22us) write_op.cc:454] Released schema lock
0929 20:47:46.069462 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0929 20:47:46.069539 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0929 20:47:46.069542 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":692169,"Hive Metastore.run_cpu_time_us":297,"Hive Metastore.run_wall_time_us":127792,"child_traces":[["op",{"apply.queue_time_us":100,"apply.run_cpu_time_us":602,"apply.run_wall_time_us":602,"num_ops":2,"prepare.queue_time_us":55,"prepare.run_cpu_time_us":921,"prepare.run_wall_time_us":920,"raft.queue_time_us":1011,"raft.run_cpu_time_us":465,"raft.run_wall_time_us":757,"replication_time_us":5070,"spinlock_wait_cycles":991104}]]}
I20260929 20:47:46.073417 1388 tablet_bootstrap.cc:492] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298: Bootstrap starting.
I20260929 20:47:46.075255 1388 tablet_bootstrap.cc:654] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:46.075316 654 catalog_manager.cc:5495] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 reported cstate change: term changed from 0 to 1, leader changed from <none> to cac815b9c12e40c6b63d4af82248f298 (127.0.97.2). New cstate: current_term: 1 leader_uuid: "cac815b9c12e40c6b63d4af82248f298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } health_report { overall_health: UNKNOWN } } }
I20260929 20:47:46.075985 654 catalog_manager.cc:5495] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 reported cstate change: term changed from 0 to 1, leader changed from <none> to cac815b9c12e40c6b63d4af82248f298 (127.0.97.2). New cstate: current_term: 1 leader_uuid: "cac815b9c12e40c6b63d4af82248f298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } health_report { overall_health: UNKNOWN } } }
I20260929 20:47:46.082917 1400 raft_consensus.cc:686] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 LEADER]: Becoming Leader. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Running, Role: LEADER
I20260929 20:47:46.083468 1400 consensus_queue.cc:237] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:46.087301 1388 tablet_bootstrap.cc:492] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298: No bootstrap required, opened a new log
I20260929 20:47:46.087507 1388 ts_tablet_manager.cc:1397] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.001s
I20260929 20:47:46.088268 1388 raft_consensus.cc:348] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:46.088557 1388 raft_consensus.cc:374] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:46.088655 1388 raft_consensus.cc:729] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Initialized, Role: FOLLOWER
I20260929 20:47:46.090281 1388 consensus_queue.cc:260] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:46.093200 1388 ts_tablet_manager.cc:1428] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260929 20:47:46.099032 652 catalog_manager.cc:2148] created HMS catalog entry for table [id=f825b412c46842ec967df7bb7d8f9474]
I20260929 20:47:46.107205 1114 tablet_service.cc:1467] Processing CreateTablet for tablet 061c178e50924b51a92542bea703b24c (DEFAULT_TABLE table=default.table_556595b54fef48f99c72c48e730495e9 [id=b17facaee1924496b1beaf3d695c11ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.107647 1114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 061c178e50924b51a92542bea703b24c. 1 dirs total, 0 dirs full, 0 dirs failed
F20260929 20:47:46.111579 875 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_4cf3799777554d47b5dbb898a6d368b9
*** Check failure stack trace: ***
@ 0x7fca59dc29ad google::LogMessage::Fail() at ??:0
I20260929 20:47:46.115195 1258 tablet_service.cc:1467] Processing CreateTablet for tablet 061c178e50924b51a92542bea703b24c (DEFAULT_TABLE table=default.table_556595b54fef48f99c72c48e730495e9 [id=b17facaee1924496b1beaf3d695c11ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.115273 971 tablet_service.cc:1467] Processing CreateTablet for tablet 061c178e50924b51a92542bea703b24c (DEFAULT_TABLE table=default.table_556595b54fef48f99c72c48e730495e9 [id=b17facaee1924496b1beaf3d695c11ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.115653 971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 061c178e50924b51a92542bea703b24c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:46.115672 1258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 061c178e50924b51a92542bea703b24c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:46.125593 1388 tablet_bootstrap.cc:492] T 061c178e50924b51a92542bea703b24c P cac815b9c12e40c6b63d4af82248f298: Bootstrap starting.
I20260929 20:47:46.125845 654 catalog_manager.cc:5495] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1). New cstate: current_term: 1 leader_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } health_report { overall_health: UNKNOWN } } }
@ 0x7fca59dc588a google::LogMessage::SendToLog() at ??:0
I20260929 20:47:46.127676 1388 tablet_bootstrap.cc:654] T 061c178e50924b51a92542bea703b24c P cac815b9c12e40c6b63d4af82248f298: Neither blocks nor log segments found. Creating new log.
@ 0x7fca59dc24de google::LogMessage::Flush() at ??:0
I20260929 20:47:46.133786 1399 raft_consensus.cc:482] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:46.134032 1399 raft_consensus.cc:504] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:46.136181 1399 leader_election.cc:288] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:46.156863 1135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9201e11b19f84f0db4e2b132a1b60ff8" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298" is_pre_election: true
I20260929 20:47:46.157140 1135 raft_consensus.cc:2454] T 9201e11b19f84f0db4e2b132a1b60ff8 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 0.
I20260929 20:47:46.157750 1212 leader_election.cc:302] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298; no voters:
I20260929 20:47:46.158990 1399 raft_consensus.cc:2781] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 20:47:46.159124 1399 raft_consensus.cc:482] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:47:46.159224 1399 raft_consensus.cc:3037] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.162931 1399 raft_consensus.cc:504] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:46.164412 1135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9201e11b19f84f0db4e2b132a1b60ff8" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298"
I20260929 20:47:46.164700 1135 raft_consensus.cc:3037] T 9201e11b19f84f0db4e2b132a1b60ff8 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.169503 1399 leader_election.cc:288] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 election: Requested vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:46.171173 1135 raft_consensus.cc:2454] T 9201e11b19f84f0db4e2b132a1b60ff8 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 1.
I20260929 20:47:46.171623 1212 leader_election.cc:302] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298; no voters:
I20260929 20:47:46.173260 652 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35484 (request call id 11) took 1241 ms. Trace:
I20260929 20:47:46.173408 652 rpcz_store.cc:276] 0929 20:47:44.931561 (+ 0us) service_pool.cc:170] Inserting onto call queue
0929 20:47:44.932344 (+ 783us) service_pool.cc:227] Handling call
0929 20:47:45.242790 (+310446us) catalog_manager.cc:2075] Acquired catalog manager lock
0929 20:47:45.243220 (+ 430us) catalog_manager.cc:2121] Created new table and tablet info
0929 20:47:46.098975 (+855755us) catalog_manager.cc:2147] Created new table in HMS catalog
0929 20:47:46.172988 (+ 74013us) catalog_manager.cc:2181] Wrote table and tablets to system table
0929 20:47:46.173112 (+ 124us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0929 20:47:46.173216 (+ 104us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0929 20:47:46.101554 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0929 20:47:46.101663 (+ 109us) write_op.cc:432] Acquiring schema lock in shared mode
0929 20:47:46.101669 (+ 6us) write_op.cc:435] Acquired schema lock
0929 20:47:46.101671 (+ 2us) tablet.cc:660] Decoding operations
0929 20:47:46.101762 (+ 91us) write_op.cc:620] Acquiring the partition lock for write op
0929 20:47:46.101773 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0929 20:47:46.101778 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0929 20:47:46.101856 (+ 78us) tablet.cc:699] Row locks acquired
0929 20:47:46.101859 (+ 3us) write_op.cc:260] PREPARE: finished
0929 20:47:46.101889 (+ 30us) write_op.cc:270] Start()
0929 20:47:46.101923 (+ 34us) write_op.cc:276] Timestamp: P: 1790714866101856 usec, L: 0
0929 20:47:46.101926 (+ 3us) op_driver.cc:331] REPLICATION: starting
0929 20:47:46.102147 (+ 221us) log.cc:842] Serialized 432 byte log entry
0929 20:47:46.171690 (+ 69543us) op_driver.cc:444] REPLICATION: finished
0929 20:47:46.172383 (+ 693us) write_op.cc:301] APPLY: starting
0929 20:47:46.172407 (+ 24us) tablet.cc:1365] starting BulkCheckPresence
0929 20:47:46.172454 (+ 47us) tablet.cc:1368] finished BulkCheckPresence
0929 20:47:46.172486 (+ 32us) tablet.cc:1370] starting ApplyRowOperation cycle
0929 20:47:46.172642 (+ 156us) tablet.cc:1381] finished ApplyRowOperation cycle
0929 20:47:46.172682 (+ 40us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0929 20:47:46.172689 (+ 7us) write_op.cc:312] APPLY: finished
0929 20:47:46.172782 (+ 93us) log.cc:842] Serialized 28 byte log entry
0929 20:47:46.172871 (+ 89us) write_op.cc:489] Releasing partition, row and schema locks
0929 20:47:46.172896 (+ 25us) write_op.cc:454] Released schema lock
0929 20:47:46.172904 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0929 20:47:46.172970 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0929 20:47:46.172973 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":818748,"Hive Metastore.run_cpu_time_us":250,"Hive Metastore.run_wall_time_us":36753,"child_traces":[["op",{"apply.queue_time_us":625,"apply.run_cpu_time_us":631,"apply.run_wall_time_us":647,"num_ops":2,"prepare.queue_time_us":1867,"prepare.run_cpu_time_us":691,"prepare.run_wall_time_us":692,"raft.queue_time_us":59,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":69733}]]}
I20260929 20:47:46.180136 652 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35484:
name: "default.Table_21a7c2dce9cf486580fad3131b93aa63"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:46.181931 654 master_service.cc:946] ReplaceTablet: received request to replace tablet e0bb85fb9e3d49be88b7fa9dccec6cda from {username='slave'} at 127.0.0.1:35484
I20260929 20:47:46.189445 971 tablet_service.cc:1467] Processing CreateTablet for tablet b2f7d9dde9ad4a10b7fbda43afa142c3 (DEFAULT_TABLE table=default.table_47f3dc18fe564237bda167352f1afa08 [id=f825b412c46842ec967df7bb7d8f9474]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.189988 971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2f7d9dde9ad4a10b7fbda43afa142c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:46.191942 1391 tablet_bootstrap.cc:492] T 061c178e50924b51a92542bea703b24c P 1cb32e9690a546149d11339d371c832f: Bootstrap starting.
I20260929 20:47:46.193710 1391 tablet_bootstrap.cc:654] T 061c178e50924b51a92542bea703b24c P 1cb32e9690a546149d11339d371c832f: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:46.194896 654 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Sending DeleteTablet for 3 replicas of tablet e0bb85fb9e3d49be88b7fa9dccec6cda
I20260929 20:47:46.195345 654 catalog_manager.cc:6565] ReplaceTablet: tablet e0bb85fb9e3d49be88b7fa9dccec6cda deleted and replaced by tablet b6df369f96d54b33997ee5d4b4860df9
I20260929 20:47:46.195570 654 master_service.cc:946] ReplaceTablet: received request to replace tablet 1982dc006d3c45bcb4f843e7067929ee from {username='slave'} at 127.0.0.1:35484
I20260929 20:47:46.196111 654 master_service.cc:946] ReplaceTablet: received request to replace tablet e0bb85fb9e3d49be88b7fa9dccec6cda from {username='slave'} at 127.0.0.1:35484
@ 0x7fca59dc26d1 google::LogMessage::~LogMessage() at ??:0
I20260929 20:47:46.198855 970 tablet_service.cc:1514] Processing DeleteTablet for tablet e0bb85fb9e3d49be88b7fa9dccec6cda with delete_type TABLET_DATA_DELETED (replaced by tablet b6df369f96d54b33997ee5d4b4860df9) from {username='slave'} at 127.0.0.1:47676
I20260929 20:47:46.199167 1392 tablet_replica.cc:330] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86: stopping tablet replica
I20260929 20:47:46.199379 1392 raft_consensus.cc:2229] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 20:47:46.199579 1392 raft_consensus.cc:2258] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 20:47:46.200577 1392 ts_tablet_manager.cc:1905] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 20:47:46.203480 1399 raft_consensus.cc:2781] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 20:47:46.204866 654 master_service.cc:946] ReplaceTablet: received request to replace tablet 4ac5f47c77a842aca9f996f49f34daf0 from {username='slave'} at 127.0.0.1:35484
I20260929 20:47:46.206259 654 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35484:
name: "default.Table_e3c4d53b991348e8af5e335ad5847221"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:46.210915 1115 tablet_service.cc:1467] Processing CreateTablet for tablet b2f7d9dde9ad4a10b7fbda43afa142c3 (DEFAULT_TABLE table=default.table_47f3dc18fe564237bda167352f1afa08 [id=f825b412c46842ec967df7bb7d8f9474]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.211436 1115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2f7d9dde9ad4a10b7fbda43afa142c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:46.213088 1390 tablet_bootstrap.cc:492] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap starting.
I20260929 20:47:46.214740 1390 tablet_bootstrap.cc:654] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:46.238555 1392 ts_tablet_manager.cc:1918] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260929 20:47:46.238721 1392 log.cc:1196] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86: Deleting WAL directory at /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/wal/wals/e0bb85fb9e3d49be88b7fa9dccec6cda
I20260929 20:47:46.239113 1392 ts_tablet_manager.cc:1939] T e0bb85fb9e3d49be88b7fa9dccec6cda P d1f79a0bcbdb42fe89e7812efeae1a86: Deleting consensus metadata
I20260929 20:47:46.239195 1258 tablet_service.cc:1514] Processing DeleteTablet for tablet e0bb85fb9e3d49be88b7fa9dccec6cda with delete_type TABLET_DATA_DELETED (replaced by tablet b6df369f96d54b33997ee5d4b4860df9) from {username='slave'} at 127.0.0.1:54804
I20260929 20:47:46.239701 1393 tablet_replica.cc:330] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f: stopping tablet replica
I20260929 20:47:46.239907 1258 tablet_service.cc:1467] Processing CreateTablet for tablet b2f7d9dde9ad4a10b7fbda43afa142c3 (DEFAULT_TABLE table=default.table_47f3dc18fe564237bda167352f1afa08 [id=f825b412c46842ec967df7bb7d8f9474]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.239974 1393 raft_consensus.cc:2229] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 20:47:46.240149 645 catalog_manager.cc:4841] TS d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): tablet e0bb85fb9e3d49be88b7fa9dccec6cda (table default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]) successfully deleted
I20260929 20:47:46.240211 1393 raft_consensus.cc:2258] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 20:47:46.240513 1114 tablet_service.cc:1514] Processing DeleteTablet for tablet e0bb85fb9e3d49be88b7fa9dccec6cda with delete_type TABLET_DATA_DELETED (replaced by tablet b6df369f96d54b33997ee5d4b4860df9) from {username='slave'} at 127.0.0.1:35688
I20260929 20:47:46.240767 1394 tablet_replica.cc:330] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298: stopping tablet replica
I20260929 20:47:46.240824 1393 ts_tablet_manager.cc:1905] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 20:47:46.240979 1394 raft_consensus.cc:2229] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [term 1 LEADER]: Raft consensus shutting down.
I20260929 20:47:46.241217 1394 pending_rounds.cc:62] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298: Trying to abort 1 pending ops.
I20260929 20:47:46.241297 1394 pending_rounds.cc:68] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334768090896244736 op_type: NO_OP noop_request { }
I20260929 20:47:46.241454 1394 raft_consensus.cc:2866] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260929 20:47:46.241575 1394 raft_consensus.cc:2258] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 20:47:46.242761 1400 raft_consensus.cc:482] T 9201e11b19f84f0db4e2b132a1b60ff8 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:46.242955 1400 raft_consensus.cc:504] T 9201e11b19f84f0db4e2b132a1b60ff8 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:46.243772 1400 leader_election.cc:288] T 9201e11b19f84f0db4e2b132a1b60ff8 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:46.244010 1114 tablet_service.cc:1467] Processing CreateTablet for tablet b6df369f96d54b33997ee5d4b4860df9 (DEFAULT_TABLE table=default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.244432 1114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6df369f96d54b33997ee5d4b4860df9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:46.244654 1135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9201e11b19f84f0db4e2b132a1b60ff8" candidate_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298" is_pre_election: true
I20260929 20:47:46.245064 1135 raft_consensus.cc:2379] T 9201e11b19f84f0db4e2b132a1b60ff8 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1f79a0bcbdb42fe89e7812efeae1a86 in current term 1: Already voted for candidate 1cb32e9690a546149d11339d371c832f in this term.
I20260929 20:47:46.245265 1394 ts_tablet_manager.cc:1905] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 20:47:46.251782 1394 ts_tablet_manager.cc:1918] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260929 20:47:46.251922 1394 log.cc:1196] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298: Deleting WAL directory at /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/wal/wals/e0bb85fb9e3d49be88b7fa9dccec6cda
I20260929 20:47:46.252367 1394 ts_tablet_manager.cc:1939] T e0bb85fb9e3d49be88b7fa9dccec6cda P cac815b9c12e40c6b63d4af82248f298: Deleting consensus metadata
I20260929 20:47:46.254125 971 tablet_service.cc:1467] Processing CreateTablet for tablet b6df369f96d54b33997ee5d4b4860df9 (DEFAULT_TABLE table=default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.254547 971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6df369f96d54b33997ee5d4b4860df9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:46.255154 1393 ts_tablet_manager.cc:1918] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260929 20:47:46.255282 1393 log.cc:1196] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f: Deleting WAL directory at /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/wal/wals/e0bb85fb9e3d49be88b7fa9dccec6cda
I20260929 20:47:46.255613 1393 ts_tablet_manager.cc:1939] T e0bb85fb9e3d49be88b7fa9dccec6cda P 1cb32e9690a546149d11339d371c832f: Deleting consensus metadata
I20260929 20:47:46.256742 1258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2f7d9dde9ad4a10b7fbda43afa142c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:46.264078 646 catalog_manager.cc:4841] TS cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): tablet e0bb85fb9e3d49be88b7fa9dccec6cda (table default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]) successfully deleted
I20260929 20:47:46.264428 1399 raft_consensus.cc:686] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [term 1 LEADER]: Becoming Leader. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Running, Role: LEADER
I20260929 20:47:46.264887 1399 consensus_queue.cc:237] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:46.267727 645 catalog_manager.cc:4841] TS 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): tablet e0bb85fb9e3d49be88b7fa9dccec6cda (table default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]) successfully deleted
I20260929 20:47:46.268003 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9201e11b19f84f0db4e2b132a1b60ff8" candidate_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f" is_pre_election: true
I20260929 20:47:46.268733 924 leader_election.cc:302] T 9201e11b19f84f0db4e2b132a1b60ff8 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:47:46.269070 1391 tablet_bootstrap.cc:492] T 061c178e50924b51a92542bea703b24c P 1cb32e9690a546149d11339d371c832f: No bootstrap required, opened a new log
I20260929 20:47:46.269141 1400 raft_consensus.cc:3037] T 9201e11b19f84f0db4e2b132a1b60ff8 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.269333 1391 ts_tablet_manager.cc:1397] T 061c178e50924b51a92542bea703b24c P 1cb32e9690a546149d11339d371c832f: Time spent bootstrapping tablet: real 0.078s user 0.010s sys 0.001s
I20260929 20:47:46.270542 1391 raft_consensus.cc:348] T 061c178e50924b51a92542bea703b24c P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.270752 1391 raft_consensus.cc:374] T 061c178e50924b51a92542bea703b24c P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:46.270838 1391 raft_consensus.cc:729] T 061c178e50924b51a92542bea703b24c P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Initialized, Role: FOLLOWER
I20260929 20:47:46.272358 1400 raft_consensus.cc:2726] T 9201e11b19f84f0db4e2b132a1b60ff8 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260929 20:47:46.273547 1390 tablet_bootstrap.cc:492] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86: No bootstrap required, opened a new log
I20260929 20:47:46.273691 1390 ts_tablet_manager.cc:1397] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent bootstrapping tablet: real 0.061s user 0.002s sys 0.001s
I20260929 20:47:46.273123 1391 consensus_queue.cc:260] T 061c178e50924b51a92542bea703b24c P 1cb32e9690a546149d11339d371c832f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.274537 1390 raft_consensus.cc:348] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.274770 1390 raft_consensus.cc:374] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:46.274874 1390 raft_consensus.cc:729] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Initialized, Role: FOLLOWER
I20260929 20:47:46.275130 1390 consensus_queue.cc:260] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.276019 1391 ts_tablet_manager.cc:1428] T 061c178e50924b51a92542bea703b24c P 1cb32e9690a546149d11339d371c832f: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260929 20:47:46.277374 1391 tablet_bootstrap.cc:492] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f: Bootstrap starting.
W20260929 20:47:46.277397 1401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 20:47:46.277897 1401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 20:47:46.278069 1401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
@ 0x7fca6f960ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260929 20:47:46.279784 1391 tablet_bootstrap.cc:654] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:46.279944 1388 tablet_bootstrap.cc:492] T 061c178e50924b51a92542bea703b24c P cac815b9c12e40c6b63d4af82248f298: No bootstrap required, opened a new log
I20260929 20:47:46.280097 1388 ts_tablet_manager.cc:1397] T 061c178e50924b51a92542bea703b24c P cac815b9c12e40c6b63d4af82248f298: Time spent bootstrapping tablet: real 0.155s user 0.065s sys 0.069s
I20260929 20:47:46.281025 1258 tablet_service.cc:1467] Processing CreateTablet for tablet b6df369f96d54b33997ee5d4b4860df9 (DEFAULT_TABLE table=default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.281045 1388 raft_consensus.cc:348] T 061c178e50924b51a92542bea703b24c P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.281369 1388 raft_consensus.cc:374] T 061c178e50924b51a92542bea703b24c P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:46.281415 1258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6df369f96d54b33997ee5d4b4860df9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:46.281473 1388 raft_consensus.cc:729] T 061c178e50924b51a92542bea703b24c P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Initialized, Role: FOLLOWER
I20260929 20:47:46.281802 1388 consensus_queue.cc:260] T 061c178e50924b51a92542bea703b24c P cac815b9c12e40c6b63d4af82248f298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.282480 1388 ts_tablet_manager.cc:1428] T 061c178e50924b51a92542bea703b24c P cac815b9c12e40c6b63d4af82248f298: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 20:47:46.282781 1388 tablet_bootstrap.cc:492] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298: Bootstrap starting.
I20260929 20:47:46.284818 1388 tablet_bootstrap.cc:654] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298: Neither blocks nor log segments found. Creating new log.
W20260929 20:47:46.290105 1401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 20:47:46.290211 1401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 20:47:46.290268 1401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 20:47:46.290345 1401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 20:47:46.295470 1390 ts_tablet_manager.cc:1428] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
I20260929 20:47:46.297341 1390 tablet_bootstrap.cc:492] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap starting.
I20260929 20:47:46.299130 1390 tablet_bootstrap.cc:654] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:46.301469 1391 tablet_bootstrap.cc:492] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f: No bootstrap required, opened a new log
I20260929 20:47:46.301740 1391 ts_tablet_manager.cc:1397] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.006s
I20260929 20:47:46.302858 1391 raft_consensus.cc:348] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:46.303217 1391 raft_consensus.cc:374] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:46.303300 1391 raft_consensus.cc:729] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Initialized, Role: FOLLOWER
I20260929 20:47:46.303637 1391 consensus_queue.cc:260] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:46.305904 1391 ts_tablet_manager.cc:1428] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260929 20:47:46.308966 1401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.97.20:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.60:45599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.97.60
--webserver_port=43799
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 19:57:29 UTC on ccycloud.quasar-xxilbq.root.comops.site
build id 4681269
ASAN enabled
@ 0x7fca6f95f901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260929 20:47:46.310820 1401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 20:47:46.313380 1401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 20:47:46.319733 1390 tablet_bootstrap.cc:492] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86: No bootstrap required, opened a new log
I20260929 20:47:46.319621 990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9201e11b19f84f0db4e2b132a1b60ff8" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86"
I20260929 20:47:46.320206 991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9201e11b19f84f0db4e2b132a1b60ff8" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" is_pre_election: true
I20260929 20:47:46.321787 1391 tablet_bootstrap.cc:492] T b6df369f96d54b33997ee5d4b4860df9 P 1cb32e9690a546149d11339d371c832f: Bootstrap starting.
I20260929 20:47:46.322124 1390 ts_tablet_manager.cc:1397] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.001s
I20260929 20:47:46.323036 1390 raft_consensus.cc:348] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:46.323309 1390 raft_consensus.cc:374] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:46.323421 1390 raft_consensus.cc:729] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Initialized, Role: FOLLOWER
I20260929 20:47:46.323571 990 raft_consensus.cc:2454] T 9201e11b19f84f0db4e2b132a1b60ff8 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 1.
I20260929 20:47:46.323740 1390 consensus_queue.cc:260] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:46.324018 1391 tablet_bootstrap.cc:654] T b6df369f96d54b33997ee5d4b4860df9 P 1cb32e9690a546149d11339d371c832f: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:46.325490 1390 ts_tablet_manager.cc:1428] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260929 20:47:46.327699 1391 tablet_bootstrap.cc:492] T b6df369f96d54b33997ee5d4b4860df9 P 1cb32e9690a546149d11339d371c832f: No bootstrap required, opened a new log
I20260929 20:47:46.327895 1391 ts_tablet_manager.cc:1397] T b6df369f96d54b33997ee5d4b4860df9 P 1cb32e9690a546149d11339d371c832f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 20:47:46.328823 1390 tablet_bootstrap.cc:492] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap starting.
I20260929 20:47:46.328934 1391 raft_consensus.cc:348] T b6df369f96d54b33997ee5d4b4860df9 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.329470 1391 raft_consensus.cc:374] T b6df369f96d54b33997ee5d4b4860df9 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:46.329589 1391 raft_consensus.cc:729] T b6df369f96d54b33997ee5d4b4860df9 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Initialized, Role: FOLLOWER
I20260929 20:47:46.329918 1391 consensus_queue.cc:260] T b6df369f96d54b33997ee5d4b4860df9 P 1cb32e9690a546149d11339d371c832f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.330883 1390 tablet_bootstrap.cc:654] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:46.331398 1391 ts_tablet_manager.cc:1428] T b6df369f96d54b33997ee5d4b4860df9 P 1cb32e9690a546149d11339d371c832f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
@ 0x7fca6f96666c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260929 20:47:46.336408 1399 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:46.336648 1399 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
@ 0x7fca6f9664e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260929 20:47:46.337934 990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2639596901a04bdb8bdbf8ccae3d1a50" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" is_pre_election: true
I20260929 20:47:46.338240 990 raft_consensus.cc:2454] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 0.
I20260929 20:47:46.338718 1211 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, d1f79a0bcbdb42fe89e7812efeae1a86; no voters:
I20260929 20:47:46.339095 1447 raft_consensus.cc:2781] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 20:47:46.339236 1447 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:47:46.339393 1447 raft_consensus.cc:3037] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.340590 1388 tablet_bootstrap.cc:492] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298: No bootstrap required, opened a new log
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260929 20:47:46.340782 1388 ts_tablet_manager.cc:1397] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298: Time spent bootstrapping tablet: real 0.058s user 0.011s sys 0.041s
I20260929 20:47:46.342255 1388 raft_consensus.cc:348] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:46.342554 1388 raft_consensus.cc:374] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:46.342626 1388 raft_consensus.cc:729] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Initialized, Role: FOLLOWER
I20260929 20:47:46.343104 1400 raft_consensus.cc:482] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:46.343220 1447 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:46.343317 1400 raft_consensus.cc:504] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.344067 1400 leader_election.cc:288] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:46.344612 990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2639596901a04bdb8bdbf8ccae3d1a50" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86"
I20260929 20:47:46.344880 990 raft_consensus.cc:3037] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.342916 1388 consensus_queue.cc:260] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:46.345578 1388 ts_tablet_manager.cc:1428] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
@ 0x7fca5c9eceaf kudu::Thread::SuperviseThread() at ??:0
I20260929 20:47:46.347000 1388 tablet_bootstrap.cc:492] T b6df369f96d54b33997ee5d4b4860df9 P cac815b9c12e40c6b63d4af82248f298: Bootstrap starting.
I20260929 20:47:46.347993 1135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061c178e50924b51a92542bea703b24c" candidate_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298" is_pre_election: true
I20260929 20:47:46.348364 1135 raft_consensus.cc:2454] T 061c178e50924b51a92542bea703b24c P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1f79a0bcbdb42fe89e7812efeae1a86 in term 0.
@ 0x7fca5fcc66db start_thread at ??:0
I20260929 20:47:46.349444 1388 tablet_bootstrap.cc:654] T b6df369f96d54b33997ee5d4b4860df9 P cac815b9c12e40c6b63d4af82248f298: Neither blocks nor log segments found. Creating new log.
@ 0x7fca58c5c61f clone at ??:0
I20260929 20:47:46.352161 925 leader_election.cc:302] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cac815b9c12e40c6b63d4af82248f298, d1f79a0bcbdb42fe89e7812efeae1a86; no voters:
I20260929 20:47:46.352628 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061c178e50924b51a92542bea703b24c" candidate_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f" is_pre_election: true
I20260929 20:47:46.352855 1278 raft_consensus.cc:2454] T 061c178e50924b51a92542bea703b24c P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1f79a0bcbdb42fe89e7812efeae1a86 in term 0.
I20260929 20:47:46.353170 1400 raft_consensus.cc:2781] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 20:47:46.353268 1400 raft_consensus.cc:482] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:47:46.353381 1400 raft_consensus.cc:3037] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.353475 1399 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:46.353749 1447 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 election: Requested vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:46.354545 1134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2639596901a04bdb8bdbf8ccae3d1a50" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298"
I20260929 20:47:46.354862 1134 raft_consensus.cc:3037] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.356737 1135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2639596901a04bdb8bdbf8ccae3d1a50" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298" is_pre_election: true
I20260929 20:47:46.360616 1388 tablet_bootstrap.cc:492] T b6df369f96d54b33997ee5d4b4860df9 P cac815b9c12e40c6b63d4af82248f298: No bootstrap required, opened a new log
I20260929 20:47:46.360733 1134 raft_consensus.cc:2454] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 1.
I20260929 20:47:46.360888 1388 ts_tablet_manager.cc:1397] T b6df369f96d54b33997ee5d4b4860df9 P cac815b9c12e40c6b63d4af82248f298: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260929 20:47:46.361331 1212 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298; no voters:
I20260929 20:47:46.361738 1447 raft_consensus.cc:2781] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 20:47:46.361925 1447 raft_consensus.cc:686] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [term 1 LEADER]: Becoming Leader. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Running, Role: LEADER
I20260929 20:47:46.361948 1388 raft_consensus.cc:348] T b6df369f96d54b33997ee5d4b4860df9 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.362272 1388 raft_consensus.cc:374] T b6df369f96d54b33997ee5d4b4860df9 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:46.362438 1388 raft_consensus.cc:729] T b6df369f96d54b33997ee5d4b4860df9 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Initialized, Role: FOLLOWER
I20260929 20:47:46.362242 1447 consensus_queue.cc:237] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:46.362751 1388 consensus_queue.cc:260] T b6df369f96d54b33997ee5d4b4860df9 P cac815b9c12e40c6b63d4af82248f298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.368053 1388 ts_tablet_manager.cc:1428] T b6df369f96d54b33997ee5d4b4860df9 P cac815b9c12e40c6b63d4af82248f298: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260929 20:47:46.377035 990 raft_consensus.cc:2454] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 1.
I20260929 20:47:46.383486 1400 raft_consensus.cc:504] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.384315 1400 leader_election.cc:288] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 1 election: Requested vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:46.385175 1390 tablet_bootstrap.cc:492] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86: No bootstrap required, opened a new log
I20260929 20:47:46.385188 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061c178e50924b51a92542bea703b24c" candidate_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f"
I20260929 20:47:46.385319 1390 ts_tablet_manager.cc:1397] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent bootstrapping tablet: real 0.057s user 0.004s sys 0.025s
I20260929 20:47:46.385401 1278 raft_consensus.cc:3037] T 061c178e50924b51a92542bea703b24c P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Advancing to term 1
W20260929 20:47:46.385761 1466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 20:47:46.386148 1390 raft_consensus.cc:348] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.386386 1390 raft_consensus.cc:374] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:46.386461 1390 raft_consensus.cc:729] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Initialized, Role: FOLLOWER
I20260929 20:47:46.386670 1390 consensus_queue.cc:260] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.387981 1390 ts_tablet_manager.cc:1428] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260929 20:47:46.388232 1278 raft_consensus.cc:2454] T 061c178e50924b51a92542bea703b24c P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1f79a0bcbdb42fe89e7812efeae1a86 in term 1.
I20260929 20:47:46.388646 924 leader_election.cc:302] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, d1f79a0bcbdb42fe89e7812efeae1a86; no voters:
I20260929 20:47:46.388971 1400 raft_consensus.cc:2781] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 20:47:46.389137 1400 raft_consensus.cc:686] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 LEADER]: Becoming Leader. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Running, Role: LEADER
I20260929 20:47:46.389259 1134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061c178e50924b51a92542bea703b24c" candidate_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298"
I20260929 20:47:46.389467 1134 raft_consensus.cc:3037] T 061c178e50924b51a92542bea703b24c P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.389375 1400 consensus_queue.cc:237] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.392294 1134 raft_consensus.cc:2454] T 061c178e50924b51a92542bea703b24c P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1f79a0bcbdb42fe89e7812efeae1a86 in term 1.
I20260929 20:47:46.394186 1401 server_base.cc:1034] running on GCE node
W20260929 20:47:46.394767 1464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:46.395236 1468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 20:47:46.396248 1401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 20:47:46.404795 1401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 20:47:46.412326 1401 hybrid_clock.cc:648] HybridClock initialized: now 1790714866412198 us; error 106 us; skew 500 ppm
I20260929 20:47:46.413044 1401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 20:47:46.420192 1447 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:46.420593 1447 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:46.421394 1447 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:46.422173 990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f7d9dde9ad4a10b7fbda43afa142c3" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" is_pre_election: true
I20260929 20:47:46.422462 990 raft_consensus.cc:2454] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 0.
I20260929 20:47:46.422891 1134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f7d9dde9ad4a10b7fbda43afa142c3" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298" is_pre_election: true
I20260929 20:47:46.423183 1134 raft_consensus.cc:2454] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 0.
I20260929 20:47:46.423688 1212 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298; no voters:
I20260929 20:47:46.424013 1447 raft_consensus.cc:2781] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 20:47:46.424108 1447 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:47:46.424199 1447 raft_consensus.cc:3037] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.427520 1447 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:46.428835 990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f7d9dde9ad4a10b7fbda43afa142c3" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86"
I20260929 20:47:46.429088 990 raft_consensus.cc:3037] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.432565 990 raft_consensus.cc:2454] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 1.
I20260929 20:47:46.433111 1211 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, d1f79a0bcbdb42fe89e7812efeae1a86; no voters:
I20260929 20:47:46.433454 1399 raft_consensus.cc:2781] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 20:47:46.437315 1134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f7d9dde9ad4a10b7fbda43afa142c3" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298"
I20260929 20:47:46.437541 1134 raft_consensus.cc:3037] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.440392 1134 raft_consensus.cc:2454] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 1.
I20260929 20:47:46.441188 1447 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 election: Requested vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:46.448916 1399 raft_consensus.cc:686] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [term 1 LEADER]: Becoming Leader. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Running, Role: LEADER
I20260929 20:47:46.449293 1399 consensus_queue.cc:237] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:46.455713 1401 webserver.cc:460] Webserver started at http://127.0.97.60:43799/ using document root <none> and password file <none>
I20260929 20:47:46.456923 1401 fs_manager.cc:359] Metadata directory not provided
I20260929 20:47:46.457084 1401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 20:47:46.464591 1401 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.005s
I20260929 20:47:46.465181 1399 raft_consensus.cc:482] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:46.465359 1399 raft_consensus.cc:504] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:46.465977 1399 leader_election.cc:288] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:46.466687 1134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6820644bb2a745679d959d8ac6854014" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298" is_pre_election: true
I20260929 20:47:46.466935 1134 raft_consensus.cc:2454] T 6820644bb2a745679d959d8ac6854014 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 0.
I20260929 20:47:46.467481 1212 leader_election.cc:302] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298; no voters:
I20260929 20:47:46.467482 990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6820644bb2a745679d959d8ac6854014" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" is_pre_election: true
I20260929 20:47:46.467724 990 raft_consensus.cc:2454] T 6820644bb2a745679d959d8ac6854014 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 0.
I20260929 20:47:46.467832 1399 raft_consensus.cc:2781] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 20:47:46.467922 1399 raft_consensus.cc:482] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:47:46.468035 1399 raft_consensus.cc:3037] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.470777 1399 raft_consensus.cc:504] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:46.471738 990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6820644bb2a745679d959d8ac6854014" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86"
I20260929 20:47:46.471935 990 raft_consensus.cc:3037] T 6820644bb2a745679d959d8ac6854014 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.474810 990 raft_consensus.cc:2454] T 6820644bb2a745679d959d8ac6854014 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 1.
I20260929 20:47:46.475291 1211 leader_election.cc:302] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, d1f79a0bcbdb42fe89e7812efeae1a86; no voters:
I20260929 20:47:46.475557 1478 raft_consensus.cc:2781] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 20:47:46.475740 1478 raft_consensus.cc:686] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [term 1 LEADER]: Becoming Leader. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Running, Role: LEADER
I20260929 20:47:46.476029 1478 consensus_queue.cc:237] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:46.476333 1134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6820644bb2a745679d959d8ac6854014" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298"
I20260929 20:47:46.476646 1134 raft_consensus.cc:3037] T 6820644bb2a745679d959d8ac6854014 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.480094 1134 raft_consensus.cc:2454] T 6820644bb2a745679d959d8ac6854014 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 1.
I20260929 20:47:46.480805 1399 leader_election.cc:288] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 election: Requested vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:46.482900 1482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 20:47:46.486275 1401 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.006s sys 0.011s
I20260929 20:47:46.486485 1401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data,/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/wal
uuid: "bfbad3ef4593415191fa0d993866d0ea"
format_stamp: "Formatted at 2026-09-29 20:47:42 on dist-test-slave-j0bz"
I20260929 20:47:46.487911 1401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 20:47:46.506404 1401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 20:47:46.507454 1401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 20:47:46.507733 1401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 20:47:46.516131 1397 consensus_queue.cc:1035] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 20:47:46.520525 1400 raft_consensus.cc:482] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:46.520756 1400 raft_consensus.cc:504] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.522550 1134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6df369f96d54b33997ee5d4b4860df9" candidate_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298" is_pre_election: true
I20260929 20:47:46.523030 1134 raft_consensus.cc:2454] T b6df369f96d54b33997ee5d4b4860df9 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1f79a0bcbdb42fe89e7812efeae1a86 in term 0.
I20260929 20:47:46.523700 925 leader_election.cc:302] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cac815b9c12e40c6b63d4af82248f298, d1f79a0bcbdb42fe89e7812efeae1a86; no voters:
I20260929 20:47:46.524155 1431 raft_consensus.cc:2781] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 20:47:46.524292 1431 raft_consensus.cc:482] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:47:46.524415 1431 raft_consensus.cc:3037] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.528796 1400 leader_election.cc:288] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:46.529858 1277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6df369f96d54b33997ee5d4b4860df9" candidate_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f" is_pre_election: true
I20260929 20:47:46.530298 1277 raft_consensus.cc:2454] T b6df369f96d54b33997ee5d4b4860df9 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1f79a0bcbdb42fe89e7812efeae1a86 in term 0.
I20260929 20:47:46.528314 1431 raft_consensus.cc:504] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.532341 1134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6df369f96d54b33997ee5d4b4860df9" candidate_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298"
I20260929 20:47:46.532660 1134 raft_consensus.cc:3037] T b6df369f96d54b33997ee5d4b4860df9 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.535995 1134 raft_consensus.cc:2454] T b6df369f96d54b33997ee5d4b4860df9 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1f79a0bcbdb42fe89e7812efeae1a86 in term 1.
I20260929 20:47:46.536456 925 leader_election.cc:302] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cac815b9c12e40c6b63d4af82248f298, d1f79a0bcbdb42fe89e7812efeae1a86; no voters:
I20260929 20:47:46.536814 1400 raft_consensus.cc:2781] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 20:47:46.537480 1400 raft_consensus.cc:686] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 LEADER]: Becoming Leader. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Running, Role: LEADER
I20260929 20:47:46.537776 1400 consensus_queue.cc:237] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.539487 1277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6df369f96d54b33997ee5d4b4860df9" candidate_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f"
I20260929 20:47:46.539744 1277 raft_consensus.cc:3037] T b6df369f96d54b33997ee5d4b4860df9 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:46.543458 1277 raft_consensus.cc:2454] T b6df369f96d54b33997ee5d4b4860df9 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1f79a0bcbdb42fe89e7812efeae1a86 in term 1.
I20260929 20:47:46.544100 1431 leader_election.cc:288] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 1 election: Requested vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:46.559978 1397 consensus_queue.cc:1035] T 95476e3b89fe467ea3f151400dcc4f76 P cac815b9c12e40c6b63d4af82248f298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260929 20:47:46.576938 1401 rpc_server.cc:307] RPC server started. Bound to: 127.0.97.60:45599
I20260929 20:47:46.576905 1541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.97.60:45599 every 8 connection(s)
I20260929 20:47:46.579020 1401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data/info.pb
I20260929 20:47:46.587283 1542 sys_catalog.cc:263] Verifying existing consensus state
I20260929 20:47:46.588912 388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu as pid 1401
I20260929 20:47:46.593065 1542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: Bootstrap starting.
I20260929 20:47:46.608371 1542 log.cc:824] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: Log is configured to *not* fsync() on all Append() calls
I20260929 20:47:46.630478 1542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 20:47:46.631186 1542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: Bootstrap complete.
I20260929 20:47:46.643000 1542 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:46.644886 1542 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfbad3ef4593415191fa0d993866d0ea, State: Initialized, Role: FOLLOWER
I20260929 20:47:46.645665 1542 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:46.646519 1546 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:46.646940 1546 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:46.647984 1542 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: configured and running, proceeding with master startup.
I20260929 20:47:46.661827 1561 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:47:46.662010 1561 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: loading cluster ID for follower catalog manager: success
I20260929 20:47:46.665915 1561 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: acquiring CA information for follower catalog manager: success
I20260929 20:47:46.669440 1561 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 20:47:46.704478 1332 connection.cc:535] client connection to 127.0.97.62:46175 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 20:47:46.705549 1332 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.97.62:46175)
W20260929 20:47:46.706252 1332 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.97.62:46175)
W20260929 20:47:46.706703 1332 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.97.62:46175)
W20260929 20:47:46.707441 1376 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 20:47:46.705186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.97.62:46175 (local address 127.0.97.3:47147)
0929 20:47:46.705500 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0929 20:47:46.705521 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 20:47:46.705635 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 20:47:46.706545 (+ 910us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":161}
W20260929 20:47:46.708225 1563 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 20:47:46.659425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.97.62:46175 (local address 127.0.0.1:35492)
0929 20:47:46.673445 (+ 14020us) negotiation.cc:107] Waiting for socket to connect
0929 20:47:46.673488 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 20:47:46.673697 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 20:47:46.706926 (+ 33229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":13706,"thread_start_us":12943,"threads_started":1}
W20260929 20:47:46.709239 1409 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 20:47:46.705615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.97.62:46175 (local address 127.0.97.2:40275)
0929 20:47:46.707525 (+ 1910us) negotiation.cc:107] Waiting for socket to connect
0929 20:47:46.707548 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 20:47:46.707685 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 20:47:46.708514 (+ 829us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1760}
W20260929 20:47:46.714265 1332 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.97.62:46175)
I20260929 20:47:46.708909 1401 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107),
I20260929 20:47:46.719947 1401 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:47:46.733119 1566 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:47:46.733377 1566 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:46.734138 1566 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
I20260929 20:47:46.737900 1397 consensus_queue.cc:1035] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 20:47:46.761235 1517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc99fb1b059842088a50ba9c56fda6c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 33 } } ignore_live_leader: false dest_uuid: "bfbad3ef4593415191fa0d993866d0ea" is_pre_election: true
I20260929 20:47:46.762033 1517 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc99fb1b059842088a50ba9c56fda6c1 in term 2.
I20260929 20:47:46.764112 717 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bfbad3ef4593415191fa0d993866d0ea, cc99fb1b059842088a50ba9c56fda6c1; no voters:
I20260929 20:47:46.764735 1566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260929 20:47:46.764885 1566 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:47:46.765012 1566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 3 FOLLOWER]: Advancing to term 4
I20260929 20:47:46.770025 1566 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:46.771924 1517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc99fb1b059842088a50ba9c56fda6c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 33 } } ignore_live_leader: false dest_uuid: "bfbad3ef4593415191fa0d993866d0ea"
I20260929 20:47:46.772226 1517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 2 FOLLOWER]: Advancing to term 4
I20260929 20:47:46.778316 1566 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 4 election: Requested vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:47:46.785977 716 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.97.62:46175, user_credentials={real_user=slave}} blocked reactor thread for 48797.7us
W20260929 20:47:46.787304 716 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:47:46.787997 1517 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc99fb1b059842088a50ba9c56fda6c1 in term 4.
W20260929 20:47:46.789608 716 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:47:46.789966 1036 heartbeater.cc:347] Connected to a master server at 127.0.97.60:45599
I20260929 20:47:46.790225 717 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bfbad3ef4593415191fa0d993866d0ea, cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8
I20260929 20:47:46.790550 1566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260929 20:47:46.792217 1566 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 4 LEADER]: Becoming Leader. State: Replica: cc99fb1b059842088a50ba9c56fda6c1, State: Running, Role: LEADER
I20260929 20:47:46.792790 1566 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 3.33, Last appended by leader: 33, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:46.794385 1397 consensus_queue.cc:1035] T c4a57da4e5c24803b7aa3f61c7191e38 P cac815b9c12e40c6b63d4af82248f298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 20:47:46.796540 1573 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc99fb1b059842088a50ba9c56fda6c1. Latest consensus state: current_term: 4 leader_uuid: "cc99fb1b059842088a50ba9c56fda6c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:46.796898 1573 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: This master's current role is: LEADER
I20260929 20:47:46.804994 1500 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" instance_seqno: 1790714863424856) as {username='slave'} at 127.0.97.1:58235; Asking this server to re-register.
I20260929 20:47:46.807211 1036 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:46.810803 1499 ts_manager.cc:194] Registered new tserver with Master: d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657)
I20260929 20:47:46.812788 1323 heartbeater.cc:347] Connected to a master server at 127.0.97.60:45599
I20260929 20:47:46.813015 1323 heartbeater.cc:503] Master 127.0.97.60:45599 was elected leader, sending a full tablet report...
I20260929 20:47:46.816345 1576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 20:47:46.820359 1181 heartbeater.cc:347] Connected to a master server at 127.0.97.60:45599
I20260929 20:47:46.820523 1576 catalog_manager.cc:669] Loaded metadata for table default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]
I20260929 20:47:46.821154 1576 catalog_manager.cc:669] Loaded metadata for table default.table_5b5c251fd82449869826c01b42a62f31 [id=3842db9126434978a3631262acd85d32]
I20260929 20:47:46.821550 1576 catalog_manager.cc:669] Loaded metadata for table default.table_0b803212f31d45419130e1da10d1a620 [id=796a2c5810ac4950a0f9499208fad0e9]
I20260929 20:47:46.822264 1576 catalog_manager.cc:669] Loaded metadata for table default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f]
I20260929 20:47:46.822726 1576 catalog_manager.cc:669] Loaded metadata for table default.table_556595b54fef48f99c72c48e730495e9 [id=b17facaee1924496b1beaf3d695c11ab]
I20260929 20:47:46.822690 1500 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "1cb32e9690a546149d11339d371c832f" instance_seqno: 1790714864581790) as {username='slave'} at 127.0.97.3:39163; Asking this server to re-register.
I20260929 20:47:46.823230 1576 catalog_manager.cc:669] Loaded metadata for table default.table_f02a75c1e46d472d90334b4340d99e7b [id=dd1c5cbd5c6c40fc9fb5cc1dbe7026b0]
I20260929 20:47:46.823680 1576 catalog_manager.cc:669] Loaded metadata for table default.table_a5aa493381b543419904f1429ffc19f2 [id=e7f4bcce9df4454a81f6e98a5b2e69e5]
I20260929 20:47:46.824138 1576 catalog_manager.cc:669] Loaded metadata for table default.table_952135bfbb564f519938b6b206b1d463 [id=f04d505b1ef9453780f36b454bc22f46]
I20260929 20:47:46.824610 1576 catalog_manager.cc:669] Loaded metadata for table default.table_47f3dc18fe564237bda167352f1afa08 [id=f825b412c46842ec967df7bb7d8f9474]
I20260929 20:47:46.824978 1323 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:46.826234 1500 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" instance_seqno: 1790714863999745) as {username='slave'} at 127.0.97.2:45107; Asking this server to re-register.
I20260929 20:47:46.826969 1499 ts_manager.cc:194] Registered new tserver with Master: 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:46.827497 1576 tablet_loader.cc:96] loaded metadata for tablet 061c178e50924b51a92542bea703b24c (table default.table_556595b54fef48f99c72c48e730495e9 [id=b17facaee1924496b1beaf3d695c11ab])
I20260929 20:47:46.828399 1576 tablet_loader.cc:96] loaded metadata for tablet 2639596901a04bdb8bdbf8ccae3d1a50 (table default.table_5b5c251fd82449869826c01b42a62f31 [id=3842db9126434978a3631262acd85d32])
I20260929 20:47:46.829082 1576 tablet_loader.cc:96] loaded metadata for tablet 53c949bb99444778a9f15a781596e986 (table default.table_a5aa493381b543419904f1429ffc19f2 [id=e7f4bcce9df4454a81f6e98a5b2e69e5])
I20260929 20:47:46.829209 1181 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:46.829459 1576 tablet_loader.cc:96] loaded metadata for tablet 6820644bb2a745679d959d8ac6854014 (table default.table_f02a75c1e46d472d90334b4340d99e7b [id=dd1c5cbd5c6c40fc9fb5cc1dbe7026b0])
I20260929 20:47:46.829794 1576 tablet_loader.cc:96] loaded metadata for tablet 9201e11b19f84f0db4e2b132a1b60ff8 (table default.table_0b803212f31d45419130e1da10d1a620 [id=796a2c5810ac4950a0f9499208fad0e9])
I20260929 20:47:46.830180 1576 tablet_loader.cc:96] loaded metadata for tablet 95476e3b89fe467ea3f151400dcc4f76 (table default.table_952135bfbb564f519938b6b206b1d463 [id=f04d505b1ef9453780f36b454bc22f46])
I20260929 20:47:46.830542 1576 tablet_loader.cc:96] loaded metadata for tablet b2f7d9dde9ad4a10b7fbda43afa142c3 (table default.table_47f3dc18fe564237bda167352f1afa08 [id=f825b412c46842ec967df7bb7d8f9474])
I20260929 20:47:46.830884 1499 ts_manager.cc:194] Registered new tserver with Master: cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:46.830909 1576 tablet_loader.cc:96] loaded metadata for tablet b6df369f96d54b33997ee5d4b4860df9 (table default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a])
I20260929 20:47:46.831228 1576 tablet_loader.cc:96] loaded metadata for tablet c4a57da4e5c24803b7aa3f61c7191e38 (table default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f])
I20260929 20:47:46.831768 1576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 20:47:46.832994 1576 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:47:46.833060 1576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 20:47:46.834286 1576 catalog_manager.cc:1504] Loading token signing keys...
I20260929 20:47:46.835218 1576 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: Loaded TSK: 0
I20260929 20:47:46.836316 1576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 20:47:46.837213 1576 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260929 20:47:46.838028 1576 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260929 20:47:46.849465 731 master_service.cc:946] ReplaceTablet: received request to replace tablet 2639596901a04bdb8bdbf8ccae3d1a50 from {username='slave'} at 127.0.0.1:37468
I20260929 20:47:46.849568 730 master_service.cc:946] ReplaceTablet: received request to replace tablet b2f7d9dde9ad4a10b7fbda43afa142c3 from {username='slave'} at 127.0.0.1:37468
I20260929 20:47:46.849906 728 master_service.cc:946] ReplaceTablet: received request to replace tablet b2f7d9dde9ad4a10b7fbda43afa142c3 from {username='slave'} at 127.0.0.1:37468
I20260929 20:47:46.849726 729 master_service.cc:946] ReplaceTablet: received request to replace tablet b2f7d9dde9ad4a10b7fbda43afa142c3 from {username='slave'} at 127.0.0.1:37468
I20260929 20:47:46.851462 722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37468:
table { table_name: "default.Table_556595b54fef48f99c72c48e730495e9" } modify_external_catalogs: true
I20260929 20:47:46.857344 725 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37468:
table { table_name: "default.Table_47f3dc18fe564237bda167352f1afa08" } modify_external_catalogs: true
I20260929 20:47:46.859383 722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37468:
table { table_name: "default.Table_556595b54fef48f99c72c48e730495e9" } modify_external_catalogs: true
I20260929 20:47:46.859688 725 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37468:
table { table_name: "default.Table_47f3dc18fe564237bda167352f1afa08" } modify_external_catalogs: true
I20260929 20:47:46.850694 727 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37468:
name: "default.Table_031408b68a614af6b6ef8a7fe493dd75"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:46.856338 726 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37468:
name: "default.Table_0d85111ed1f34848b4c76ba1a254e00f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:46.851027 724 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37468:
name: "default.Table_be3c404093574a04aea5256b4ecba3bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:46.856976 723 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37468:
name: "default.Table_57d2f2bbf5c3465ba1591d6a2b6fbbc0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:46.865272 1517 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 4 FOLLOWER]: Refusing update from remote peer cc99fb1b059842088a50ba9c56fda6c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 35. (index mismatch)
I20260929 20:47:46.866259 1573 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 5, Time since last communication: 0.000s
F20260929 20:47:46.870008 905 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_4cf3799777554d47b5dbb898a6d368b9
*** Check failure stack trace: ***
@ 0x7f1bd99839ad google::LogMessage::Fail() at ??:0
W20260929 20:47:46.874634 716 consensus_peers.cc:487] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 -> Peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Couldn't send request to peer b32aaf3821ff4ad09bcbfe11ff2e03c8. Status: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7f1bd998688a google::LogMessage::SendToLog() at ??:0
@ 0x7f1bd99834de google::LogMessage::Flush() at ??:0
@ 0x7f1bd99836d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f1bef521ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f1bef520901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260929 20:47:46.899498 1546 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc99fb1b059842088a50ba9c56fda6c1. Latest consensus state: current_term: 4 leader_uuid: "cc99fb1b059842088a50ba9c56fda6c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:46.899912 1546 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:46.899973 1582 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "cc99fb1b059842088a50ba9c56fda6c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:46.900465 1582 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:46.903441 1582 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "cc99fb1b059842088a50ba9c56fda6c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:46.903780 1582 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:46.907235 731 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: Sending DeleteTablet for 3 replicas of tablet 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:47:46.907998 730 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: Sending DeleteTablet for 3 replicas of tablet b2f7d9dde9ad4a10b7fbda43afa142c3
@ 0x7f1bef52766c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260929 20:47:46.913534 731 catalog_manager.cc:6565] ReplaceTablet: tablet 2639596901a04bdb8bdbf8ccae3d1a50 deleted and replaced by tablet c2861c34ac464b9993b200df3fc201a1
I20260929 20:47:46.913757 730 catalog_manager.cc:6565] ReplaceTablet: tablet b2f7d9dde9ad4a10b7fbda43afa142c3 deleted and replaced by tablet 5e20f360f9b74f3f929b7acfdf82671f
I20260929 20:47:46.913931 730 master_service.cc:946] ReplaceTablet: received request to replace tablet 53c949bb99444778a9f15a781596e986 from {username='slave'} at 127.0.0.1:37468
I20260929 20:47:46.940256 731 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37468:
name: "default.Table_55d763f6ec744322a43d80efeb51983e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:46.943110 729 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37468:
table { table_name: "default.Table_5b5c251fd82449869826c01b42a62f31" } modify_external_catalogs: true
I20260929 20:47:46.943518 729 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37468:
table { table_name: "default.Table_5b5c251fd82449869826c01b42a62f31" } modify_external_catalogs: true
I20260929 20:47:46.943856 728 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37468:
table { table_name: "default.Table_f02a75c1e46d472d90334b4340d99e7b" } modify_external_catalogs: true
I20260929 20:47:46.944144 728 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37468:
table { table_name: "default.Table_f02a75c1e46d472d90334b4340d99e7b" } modify_external_catalogs: true
I20260929 20:47:46.953428 1566 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "cc99fb1b059842088a50ba9c56fda6c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:46.953783 1566 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: This master's current role is: LEADER
@ 0x7f1bef5274e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260929 20:47:46.959030 730 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: Sending DeleteTablet for 3 replicas of tablet 53c949bb99444778a9f15a781596e986
I20260929 20:47:46.959506 730 catalog_manager.cc:6565] ReplaceTablet: tablet 53c949bb99444778a9f15a781596e986 deleted and replaced by tablet 9780cc79845d46d780893950ca742c02
I20260929 20:47:46.959961 730 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37468:
name: "default.Table_e3c4d53b991348e8af5e335ad5847221"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260929 20:47:46.962028 1114 tablet_service.cc:1514] Processing DeleteTablet for tablet 2639596901a04bdb8bdbf8ccae3d1a50 with delete_type TABLET_DATA_DELETED (replaced by tablet c2861c34ac464b9993b200df3fc201a1) from {username='slave'} at 127.0.0.1:35690
I20260929 20:47:46.962626 1115 tablet_service.cc:1514] Processing DeleteTablet for tablet b2f7d9dde9ad4a10b7fbda43afa142c3 with delete_type TABLET_DATA_DELETED (replaced by tablet 5e20f360f9b74f3f929b7acfdf82671f) from {username='slave'} at 127.0.0.1:35690
I20260929 20:47:46.962880 1116 tablet_service.cc:1514] Processing DeleteTablet for tablet 53c949bb99444778a9f15a781596e986 with delete_type TABLET_DATA_DELETED (replaced by tablet 9780cc79845d46d780893950ca742c02) from {username='slave'} at 127.0.0.1:35690
I20260929 20:47:46.962839 1113 tablet_service.cc:1467] Processing CreateTablet for tablet 5e20f360f9b74f3f929b7acfdf82671f (DEFAULT_TABLE table=default.table_47f3dc18fe564237bda167352f1afa08 [id=f825b412c46842ec967df7bb7d8f9474]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.963340 1113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e20f360f9b74f3f929b7acfdf82671f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:46.963372 1112 tablet_service.cc:1467] Processing CreateTablet for tablet c2861c34ac464b9993b200df3fc201a1 (DEFAULT_TABLE table=default.table_5b5c251fd82449869826c01b42a62f31 [id=3842db9126434978a3631262acd85d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.963864 1112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2861c34ac464b9993b200df3fc201a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:46.977815 1593 tablet_replica.cc:330] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298: stopping tablet replica
I20260929 20:47:46.978081 1593 raft_consensus.cc:2229] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 20:47:46.978353 1593 raft_consensus.cc:2258] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 20:47:46.978952 1593 ts_tablet_manager.cc:1905] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 20:47:46.980878 1596 tablet_bootstrap.cc:492] T 5e20f360f9b74f3f929b7acfdf82671f P cac815b9c12e40c6b63d4af82248f298: Bootstrap starting.
I20260929 20:47:46.982263 1258 tablet_service.cc:1514] Processing DeleteTablet for tablet 2639596901a04bdb8bdbf8ccae3d1a50 with delete_type TABLET_DATA_DELETED (replaced by tablet c2861c34ac464b9993b200df3fc201a1) from {username='slave'} at 127.0.0.1:54808
I20260929 20:47:46.982802 1112 tablet_service.cc:1467] Processing CreateTablet for tablet 9780cc79845d46d780893950ca742c02 (DEFAULT_TABLE table=default.table_a5aa493381b543419904f1429ffc19f2 [id=e7f4bcce9df4454a81f6e98a5b2e69e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.983307 1112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9780cc79845d46d780893950ca742c02. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f1bdc5adeaf kudu::Thread::SuperviseThread() at ??:0
I20260929 20:47:46.985163 1257 tablet_service.cc:1514] Processing DeleteTablet for tablet b2f7d9dde9ad4a10b7fbda43afa142c3 with delete_type TABLET_DATA_DELETED (replaced by tablet 5e20f360f9b74f3f929b7acfdf82671f) from {username='slave'} at 127.0.0.1:54808
I20260929 20:47:46.985863 1257 tablet_service.cc:1514] Processing DeleteTablet for tablet 53c949bb99444778a9f15a781596e986 with delete_type TABLET_DATA_DELETED (replaced by tablet 9780cc79845d46d780893950ca742c02) from {username='slave'} at 127.0.0.1:54808
I20260929 20:47:46.986675 1257 tablet_service.cc:1467] Processing CreateTablet for tablet 5e20f360f9b74f3f929b7acfdf82671f (DEFAULT_TABLE table=default.table_47f3dc18fe564237bda167352f1afa08 [id=f825b412c46842ec967df7bb7d8f9474]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.987085 1257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e20f360f9b74f3f929b7acfdf82671f. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f1bdf8876db start_thread at ??:0
@ 0x7f1bd881d61f clone at ??:0
I20260929 20:47:46.990063 1597 tablet_replica.cc:330] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f: stopping tablet replica
I20260929 20:47:46.990262 1597 raft_consensus.cc:2229] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [term 1 LEADER]: Raft consensus shutting down.
I20260929 20:47:46.990693 1597 pending_rounds.cc:62] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f: Trying to abort 1 pending ops.
I20260929 20:47:46.990896 1596 tablet_bootstrap.cc:654] T 5e20f360f9b74f3f929b7acfdf82671f P cac815b9c12e40c6b63d4af82248f298: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:46.990798 1597 pending_rounds.cc:68] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334768092622974976 op_type: NO_OP noop_request { }
I20260929 20:47:46.991020 1597 raft_consensus.cc:2866] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260929 20:47:46.991168 1597 raft_consensus.cc:2258] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 20:47:46.993108 1258 tablet_service.cc:1467] Processing CreateTablet for tablet c2861c34ac464b9993b200df3fc201a1 (DEFAULT_TABLE table=default.table_5b5c251fd82449869826c01b42a62f31 [id=3842db9126434978a3631262acd85d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.994105 1258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2861c34ac464b9993b200df3fc201a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:46.996397 1597 ts_tablet_manager.cc:1905] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 20:47:46.997040 1256 tablet_service.cc:1467] Processing CreateTablet for tablet 9780cc79845d46d780893950ca742c02 (DEFAULT_TABLE table=default.table_a5aa493381b543419904f1429ffc19f2 [id=e7f4bcce9df4454a81f6e98a5b2e69e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 20:47:46.997077 1579 mvcc.cc:205] Tried to move back new op lower bound from 7334768094677692416 to 7334768086331482112. Current Snapshot: MvccSnapshot[applied={T|T < 7334768092058107904}]
I20260929 20:47:46.997408 1256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9780cc79845d46d780893950ca742c02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:46.997416 1596 tablet_bootstrap.cc:492] T 5e20f360f9b74f3f929b7acfdf82671f P cac815b9c12e40c6b63d4af82248f298: No bootstrap required, opened a new log
I20260929 20:47:46.997566 1596 ts_tablet_manager.cc:1397] T 5e20f360f9b74f3f929b7acfdf82671f P cac815b9c12e40c6b63d4af82248f298: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.000s
I20260929 20:47:46.998524 1596 raft_consensus.cc:348] T 5e20f360f9b74f3f929b7acfdf82671f P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:46.998919 1596 raft_consensus.cc:374] T 5e20f360f9b74f3f929b7acfdf82671f P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:46.999032 1596 raft_consensus.cc:729] T 5e20f360f9b74f3f929b7acfdf82671f P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Initialized, Role: FOLLOWER
I20260929 20:47:46.999414 1596 consensus_queue.cc:260] T 5e20f360f9b74f3f929b7acfdf82671f P cac815b9c12e40c6b63d4af82248f298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:47.004050 1596 ts_tablet_manager.cc:1428] T 5e20f360f9b74f3f929b7acfdf82671f P cac815b9c12e40c6b63d4af82248f298: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260929 20:47:47.007607 1593 ts_tablet_manager.cc:1918] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260929 20:47:47.007746 1593 log.cc:1196] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298: Deleting WAL directory at /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/wal/wals/2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:47:47.007936 1596 tablet_bootstrap.cc:492] T c2861c34ac464b9993b200df3fc201a1 P cac815b9c12e40c6b63d4af82248f298: Bootstrap starting.
I20260929 20:47:47.008168 1593 ts_tablet_manager.cc:1939] T 2639596901a04bdb8bdbf8ccae3d1a50 P cac815b9c12e40c6b63d4af82248f298: Deleting consensus metadata
I20260929 20:47:47.010234 1596 tablet_bootstrap.cc:654] T c2861c34ac464b9993b200df3fc201a1 P cac815b9c12e40c6b63d4af82248f298: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:47.018338 1601 tablet_bootstrap.cc:492] T 9780cc79845d46d780893950ca742c02 P 1cb32e9690a546149d11339d371c832f: Bootstrap starting.
I20260929 20:47:47.019749 1596 tablet_bootstrap.cc:492] T c2861c34ac464b9993b200df3fc201a1 P cac815b9c12e40c6b63d4af82248f298: No bootstrap required, opened a new log
I20260929 20:47:47.019933 1596 ts_tablet_manager.cc:1397] T c2861c34ac464b9993b200df3fc201a1 P cac815b9c12e40c6b63d4af82248f298: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.008s
I20260929 20:47:47.020849 1596 raft_consensus.cc:348] T c2861c34ac464b9993b200df3fc201a1 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:47.021106 1596 raft_consensus.cc:374] T c2861c34ac464b9993b200df3fc201a1 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:47.021181 1596 raft_consensus.cc:729] T c2861c34ac464b9993b200df3fc201a1 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Initialized, Role: FOLLOWER
I20260929 20:47:47.021440 1596 consensus_queue.cc:260] T c2861c34ac464b9993b200df3fc201a1 P cac815b9c12e40c6b63d4af82248f298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:47.023413 1596 ts_tablet_manager.cc:1428] T c2861c34ac464b9993b200df3fc201a1 P cac815b9c12e40c6b63d4af82248f298: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260929 20:47:47.024770 1596 tablet_bootstrap.cc:492] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298: Bootstrap starting.
I20260929 20:47:47.027951 1596 tablet_bootstrap.cc:654] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:47.039600 1593 tablet_replica.cc:330] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298: stopping tablet replica
I20260929 20:47:47.039963 1593 raft_consensus.cc:2229] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 20:47:47.040186 1593 raft_consensus.cc:2258] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 20:47:47.041029 1593 ts_tablet_manager.cc:1905] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 20:47:47.044188 1596 tablet_bootstrap.cc:492] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298: No bootstrap required, opened a new log
I20260929 20:47:47.044327 1596 ts_tablet_manager.cc:1397] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.012s
I20260929 20:47:47.045149 1596 raft_consensus.cc:348] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:47.045406 1596 raft_consensus.cc:374] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:47.045513 1596 raft_consensus.cc:729] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Initialized, Role: FOLLOWER
I20260929 20:47:47.045737 1596 consensus_queue.cc:260] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:47.046237 1596 ts_tablet_manager.cc:1428] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 20:47:47.048759 1601 tablet_bootstrap.cc:654] T 9780cc79845d46d780893950ca742c02 P 1cb32e9690a546149d11339d371c832f: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:47.050141 1597 ts_tablet_manager.cc:1918] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260929 20:47:47.050294 1597 log.cc:1196] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f: Deleting WAL directory at /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/wal/wals/2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:47:47.050766 1597 ts_tablet_manager.cc:1939] T 2639596901a04bdb8bdbf8ccae3d1a50 P 1cb32e9690a546149d11339d371c832f: Deleting consensus metadata
I20260929 20:47:47.054554 1593 ts_tablet_manager.cc:1918] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260929 20:47:47.054702 1593 log.cc:1196] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298: Deleting WAL directory at /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/wal/wals/b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:47:47.055105 1593 ts_tablet_manager.cc:1939] T b2f7d9dde9ad4a10b7fbda43afa142c3 P cac815b9c12e40c6b63d4af82248f298: Deleting consensus metadata
I20260929 20:47:47.056068 1593 tablet_replica.cc:330] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298: stopping tablet replica
I20260929 20:47:47.056267 1593 raft_consensus.cc:2229] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 20:47:47.056445 1593 raft_consensus.cc:2258] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 20:47:47.069867 1597 tablet_replica.cc:330] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f: stopping tablet replica
I20260929 20:47:47.069885 1601 tablet_bootstrap.cc:492] T 9780cc79845d46d780893950ca742c02 P 1cb32e9690a546149d11339d371c832f: No bootstrap required, opened a new log
I20260929 20:47:47.070079 1601 ts_tablet_manager.cc:1397] T 9780cc79845d46d780893950ca742c02 P 1cb32e9690a546149d11339d371c832f: Time spent bootstrapping tablet: real 0.052s user 0.000s sys 0.005s
I20260929 20:47:47.070127 1597 raft_consensus.cc:2229] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [term 1 LEADER]: Raft consensus shutting down.
I20260929 20:47:47.070441 1597 pending_rounds.cc:62] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f: Trying to abort 1 pending ops.
I20260929 20:47:47.070525 1597 pending_rounds.cc:68] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334768092978946048 op_type: NO_OP noop_request { }
I20260929 20:47:47.070685 1597 raft_consensus.cc:2866] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260929 20:47:47.070797 1597 raft_consensus.cc:2258] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 20:47:47.071036 1601 raft_consensus.cc:348] T 9780cc79845d46d780893950ca742c02 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:47.071331 1601 raft_consensus.cc:374] T 9780cc79845d46d780893950ca742c02 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:47.071432 1601 raft_consensus.cc:729] T 9780cc79845d46d780893950ca742c02 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Initialized, Role: FOLLOWER
I20260929 20:47:47.071710 1597 ts_tablet_manager.cc:1905] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 20:47:47.072266 1593 ts_tablet_manager.cc:1905] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 20:47:47.078814 1597 ts_tablet_manager.cc:1918] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260929 20:47:47.079116 1597 log.cc:1196] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f: Deleting WAL directory at /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/wal/wals/b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:47:47.079159 1593 ts_tablet_manager.cc:1918] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260929 20:47:47.079296 1593 log.cc:1196] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298: Deleting WAL directory at /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-1/wal/wals/53c949bb99444778a9f15a781596e986
I20260929 20:47:47.079694 1593 ts_tablet_manager.cc:1939] T 53c949bb99444778a9f15a781596e986 P cac815b9c12e40c6b63d4af82248f298: Deleting consensus metadata
I20260929 20:47:47.079844 1597 ts_tablet_manager.cc:1939] T b2f7d9dde9ad4a10b7fbda43afa142c3 P 1cb32e9690a546149d11339d371c832f: Deleting consensus metadata
I20260929 20:47:47.071727 1601 consensus_queue.cc:260] T 9780cc79845d46d780893950ca742c02 P 1cb32e9690a546149d11339d371c832f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:47.083232 1601 ts_tablet_manager.cc:1428] T 9780cc79845d46d780893950ca742c02 P 1cb32e9690a546149d11339d371c832f: Time spent starting tablet: real 0.013s user 0.000s sys 0.002s
I20260929 20:47:47.084532 1601 tablet_bootstrap.cc:492] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f: Bootstrap starting.
I20260929 20:47:47.086146 1597 tablet_replica.cc:330] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f: stopping tablet replica
I20260929 20:47:47.086410 1601 tablet_bootstrap.cc:654] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:47.086540 1597 raft_consensus.cc:2229] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 20:47:47.086848 1597 raft_consensus.cc:2258] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 20:47:47.087541 1597 ts_tablet_manager.cc:1905] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 20:47:47.095176 1601 tablet_bootstrap.cc:492] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f: No bootstrap required, opened a new log
I20260929 20:47:47.095428 1601 ts_tablet_manager.cc:1397] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260929 20:47:47.096712 1601 raft_consensus.cc:348] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:47.097033 1601 raft_consensus.cc:374] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:47.097152 1601 raft_consensus.cc:729] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Initialized, Role: FOLLOWER
I20260929 20:47:47.097482 1601 consensus_queue.cc:260] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:47.099978 1601 ts_tablet_manager.cc:1428] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260929 20:47:47.101358 1601 tablet_bootstrap.cc:492] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f: Bootstrap starting.
I20260929 20:47:47.102754 1597 ts_tablet_manager.cc:1918] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260929 20:47:47.102967 1597 log.cc:1196] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f: Deleting WAL directory at /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-2/wal/wals/53c949bb99444778a9f15a781596e986
I20260929 20:47:47.103466 1597 ts_tablet_manager.cc:1939] T 53c949bb99444778a9f15a781596e986 P 1cb32e9690a546149d11339d371c832f: Deleting consensus metadata
I20260929 20:47:47.103852 1601 tablet_bootstrap.cc:654] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f: Neither blocks nor log segments found. Creating new log.
I20260929 20:47:47.106256 1601 tablet_bootstrap.cc:492] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f: No bootstrap required, opened a new log
I20260929 20:47:47.106449 1601 ts_tablet_manager.cc:1397] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260929 20:47:47.107088 1601 raft_consensus.cc:348] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:47.107331 1601 raft_consensus.cc:374] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 20:47:47.107494 1601 raft_consensus.cc:729] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Initialized, Role: FOLLOWER
I20260929 20:47:47.107770 1601 consensus_queue.cc:260] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:47.108788 1601 ts_tablet_manager.cc:1428] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260929 20:47:47.130175 1397 raft_consensus.cc:482] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:47.130432 1397 raft_consensus.cc:504] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:47.131095 1397 leader_election.cc:288] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:47.131701 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9780cc79845d46d780893950ca742c02" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f" is_pre_election: true
I20260929 20:47:47.132050 1278 raft_consensus.cc:2454] T 9780cc79845d46d780893950ca742c02 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cac815b9c12e40c6b63d4af82248f298 in term 0.
W20260929 20:47:47.132160 1069 leader_election.cc:341] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): Not found: Tablet not found: 9780cc79845d46d780893950ca742c02
I20260929 20:47:47.132552 1069 leader_election.cc:302] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298; no voters: d1f79a0bcbdb42fe89e7812efeae1a86
I20260929 20:47:47.132839 1397 raft_consensus.cc:2781] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 20:47:47.132938 1397 raft_consensus.cc:482] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:47:47.133041 1397 raft_consensus.cc:3037] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:47.134896 1397 raft_consensus.cc:504] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:47.135494 1397 leader_election.cc:288] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 election: Requested vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:47.136008 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9780cc79845d46d780893950ca742c02" candidate_uuid: "cac815b9c12e40c6b63d4af82248f298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb32e9690a546149d11339d371c832f"
I20260929 20:47:47.136246 1278 raft_consensus.cc:3037] T 9780cc79845d46d780893950ca742c02 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:47.138351 1278 raft_consensus.cc:2454] T 9780cc79845d46d780893950ca742c02 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cac815b9c12e40c6b63d4af82248f298 in term 1.
I20260929 20:47:47.138854 1069 leader_election.cc:302] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298; no voters:
I20260929 20:47:47.139174 1397 raft_consensus.cc:2781] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Leader election won for term 1
W20260929 20:47:47.139236 1069 leader_election.cc:341] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): Not found: Tablet not found: 9780cc79845d46d780893950ca742c02
I20260929 20:47:47.139758 1397 raft_consensus.cc:686] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [term 1 LEADER]: Becoming Leader. State: Replica: cac815b9c12e40c6b63d4af82248f298, State: Running, Role: LEADER
I20260929 20:47:47.140039 1397 consensus_queue.cc:237] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
W20260929 20:47:47.203541 925 consensus_peers.cc:487] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 -> Peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Couldn't send request to peer cac815b9c12e40c6b63d4af82248f298. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53c949bb99444778a9f15a781596e986. This is attempt 1: this message will repeat every 5th retry.
I20260929 20:47:47.205631 1478 consensus_queue.cc:1035] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 20:47:47.221676 1478 consensus_queue.cc:1035] T 9201e11b19f84f0db4e2b132a1b60ff8 P 1cb32e9690a546149d11339d371c832f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 20:47:47.294279 1400 consensus_queue.cc:1035] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 20:47:47.302651 1400 consensus_queue.cc:1035] T 061c178e50924b51a92542bea703b24c P d1f79a0bcbdb42fe89e7812efeae1a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 20:47:47.318593 1070 connection.cc:535] server connection from 127.0.0.1:35690 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 20:47:47.318593 1213 connection.cc:535] server connection from 127.0.0.1:54808 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 20:47:47.319005 1331 connection.cc:535] client connection to 127.0.97.61:34103 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 20:47:47.320420 1588 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 20:47:46.910665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47688 (local address 127.0.97.1:45657)
0929 20:47:46.944545 (+ 33880us) server_negotiation.cc:206] Beginning negotiation
0929 20:47:46.944554 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 20:47:46.944583 (+ 29us) server_negotiation.cc:407] Connection header received
0929 20:47:46.944674 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 20:47:46.944682 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 20:47:46.944786 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 20:47:46.945511 (+ 725us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 20:47:46.945703 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 20:47:46.947040 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 20:47:46.956934 (+ 9894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 20:47:47.021009 (+ 64075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 20:47:47.021179 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 20:47:47.022545 (+ 1366us) server_negotiation.cc:1035] Waiting for connection context
0929 20:47:47.319309 (+296764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47688: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":32205,"thread_start_us":42101,"threads_started":1}
W20260929 20:47:47.321727 1069 connection.cc:535] client connection to 127.0.97.61:34103 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 20:47:47.322422 924 connection.cc:535] client connection to 127.0.97.61:34103 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 20:47:47.323578 1617 raft_consensus.cc:482] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260929 20:47:47.323778 1211 connection.cc:535] client connection to 127.0.97.61:34103 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 20:47:47.323794 1617 raft_consensus.cc:504] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
W20260929 20:47:47.324760 924 consensus_peers.cc:487] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 -> Peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Couldn't send request to peer 1cb32e9690a546149d11339d371c832f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53c949bb99444778a9f15a781596e986. This is attempt 1: this message will repeat every 5th retry.
I20260929 20:47:47.324888 1617 leader_election.cc:288] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657)
I20260929 20:47:47.325464 1134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e20f360f9b74f3f929b7acfdf82671f" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298" is_pre_election: true
I20260929 20:47:47.325770 1134 raft_consensus.cc:2454] T 5e20f360f9b74f3f929b7acfdf82671f P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 0.
I20260929 20:47:47.326310 1212 leader_election.cc:302] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298; no voters:
I20260929 20:47:47.326591 1617 raft_consensus.cc:2781] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 20:47:47.326707 1617 raft_consensus.cc:482] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:47:47.326804 1617 raft_consensus.cc:3037] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Advancing to term 1
W20260929 20:47:47.328989 1211 leader_election.cc:341] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): Not found: Tablet not found: 5e20f360f9b74f3f929b7acfdf82671f
I20260929 20:47:47.329665 1617 raft_consensus.cc:504] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:47.330266 1617 leader_election.cc:288] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 election: Requested vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657)
I20260929 20:47:47.330709 1134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e20f360f9b74f3f929b7acfdf82671f" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298"
I20260929 20:47:47.331029 1134 raft_consensus.cc:3037] T 5e20f360f9b74f3f929b7acfdf82671f P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Advancing to term 1
W20260929 20:47:47.331027 1211 leader_election.cc:341] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): Not found: Tablet not found: 5e20f360f9b74f3f929b7acfdf82671f
I20260929 20:47:47.333552 1134 raft_consensus.cc:2454] T 5e20f360f9b74f3f929b7acfdf82671f P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 1.
I20260929 20:47:47.333949 1212 leader_election.cc:302] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298; no voters: d1f79a0bcbdb42fe89e7812efeae1a86
I20260929 20:47:47.334266 1617 raft_consensus.cc:2781] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 20:47:47.334437 1617 raft_consensus.cc:686] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [term 1 LEADER]: Becoming Leader. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Running, Role: LEADER
I20260929 20:47:47.334705 1617 consensus_queue.cc:237] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:47.378494 1617 raft_consensus.cc:482] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:47.378763 1617 raft_consensus.cc:504] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:47.379763 1617 leader_election.cc:288] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:47:47.380262 1211 leader_election.cc:341] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): Not found: Tablet not found: c2861c34ac464b9993b200df3fc201a1
I20260929 20:47:47.380267 1134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2861c34ac464b9993b200df3fc201a1" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298" is_pre_election: true
I20260929 20:47:47.380595 1134 raft_consensus.cc:2454] T c2861c34ac464b9993b200df3fc201a1 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 0.
I20260929 20:47:47.381023 1212 leader_election.cc:302] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298; no voters: d1f79a0bcbdb42fe89e7812efeae1a86
I20260929 20:47:47.381328 1617 raft_consensus.cc:2781] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 20:47:47.381436 1617 raft_consensus.cc:482] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:47:47.381510 1617 raft_consensus.cc:3037] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [term 0 FOLLOWER]: Advancing to term 1
I20260929 20:47:47.383296 1617 raft_consensus.cc:504] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:47.383857 1617 leader_election.cc:288] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 election: Requested vote from peers d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:47.384369 1134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2861c34ac464b9993b200df3fc201a1" candidate_uuid: "1cb32e9690a546149d11339d371c832f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cac815b9c12e40c6b63d4af82248f298"
I20260929 20:47:47.384624 1134 raft_consensus.cc:3037] T c2861c34ac464b9993b200df3fc201a1 P cac815b9c12e40c6b63d4af82248f298 [term 0 FOLLOWER]: Advancing to term 1
W20260929 20:47:47.384761 1211 leader_election.cc:341] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): Not found: Tablet not found: c2861c34ac464b9993b200df3fc201a1
I20260929 20:47:47.386564 1134 raft_consensus.cc:2454] T c2861c34ac464b9993b200df3fc201a1 P cac815b9c12e40c6b63d4af82248f298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cb32e9690a546149d11339d371c832f in term 1.
I20260929 20:47:47.387029 1212 leader_election.cc:302] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298; no voters: d1f79a0bcbdb42fe89e7812efeae1a86
I20260929 20:47:47.387295 1617 raft_consensus.cc:2781] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 20:47:47.387470 1617 raft_consensus.cc:686] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [term 1 LEADER]: Becoming Leader. State: Replica: 1cb32e9690a546149d11339d371c832f, State: Running, Role: LEADER
I20260929 20:47:47.387751 1617 consensus_queue.cc:237] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:47.429351 1617 consensus_queue.cc:1035] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 20:47:47.437158 1616 consensus_queue.cc:1035] T 6820644bb2a745679d959d8ac6854014 P 1cb32e9690a546149d11339d371c832f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 20:47:47.469586 1631 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cc99fb1b059842088a50ba9c56fda6c1)
I20260929 20:47:47.469806 1631 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:47.471511 1631 leader_election.cc:288] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103)
W20260929 20:47:47.473138 1485 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:47:47.474684 1484 leader_election.cc:334] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
W20260929 20:47:47.474932 1485 leader_election.cc:334] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:47:47.475114 1485 leader_election.cc:302] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfbad3ef4593415191fa0d993866d0ea; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:47:47.475500 1631 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260929 20:47:47.521679 1620 consensus_queue.cc:1035] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 20:47:47.527673 1621 consensus_queue.cc:1035] T b6df369f96d54b33997ee5d4b4860df9 P d1f79a0bcbdb42fe89e7812efeae1a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 20:47:48.049120 1640 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cc99fb1b059842088a50ba9c56fda6c1)
I20260929 20:47:48.049300 1640 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:48.050103 1640 leader_election.cc:288] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103)
W20260929 20:47:48.052465 1485 leader_election.cc:334] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:47:48.052488 1069 consensus_peers.cc:487] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 -> Peer d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): Couldn't send request to peer d1f79a0bcbdb42fe89e7812efeae1a86. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9780cc79845d46d780893950ca742c02. This is attempt 1: this message will repeat every 5th retry.
W20260929 20:47:48.052675 1484 leader_election.cc:334] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
I20260929 20:47:48.052865 1484 leader_election.cc:302] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfbad3ef4593415191fa0d993866d0ea; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:47:48.053126 1640 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260929 20:47:48.306664 1397 consensus_queue.cc:1035] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 20:47:48.352705 1211 consensus_peers.cc:487] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f -> Peer d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): Couldn't send request to peer d1f79a0bcbdb42fe89e7812efeae1a86. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5e20f360f9b74f3f929b7acfdf82671f. This is attempt 1: this message will repeat every 5th retry.
W20260929 20:47:48.502699 1211 consensus_peers.cc:487] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f -> Peer d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): Couldn't send request to peer d1f79a0bcbdb42fe89e7812efeae1a86. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c2861c34ac464b9993b200df3fc201a1. This is attempt 1: this message will repeat every 5th retry.
I20260929 20:47:48.521330 1645 consensus_queue.cc:1035] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 20:47:48.568922 1646 consensus_queue.cc:1035] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 20:47:48.872381 1665 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cc99fb1b059842088a50ba9c56fda6c1)
I20260929 20:47:48.872618 1665 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:48.873337 1665 leader_election.cc:288] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103)
W20260929 20:47:48.875995 1484 leader_election.cc:334] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
W20260929 20:47:48.876205 1485 leader_election.cc:334] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:47:48.876339 1485 leader_election.cc:302] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfbad3ef4593415191fa0d993866d0ea; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:47:48.876631 1665 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260929 20:47:49.028439 1669 ts_tablet_manager.cc:927] T 9780cc79845d46d780893950ca742c02 P d1f79a0bcbdb42fe89e7812efeae1a86: Initiating tablet copy from peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:49.029554 1669 tablet_copy_client.cc:316] T 9780cc79845d46d780893950ca742c02 P d1f79a0bcbdb42fe89e7812efeae1a86: tablet copy: Beginning tablet copy session from remote peer at address 127.0.97.2:41081
I20260929 20:47:49.031133 1156 tablet_copy_service.cc:140] P cac815b9c12e40c6b63d4af82248f298: Received BeginTabletCopySession request for tablet 9780cc79845d46d780893950ca742c02 from peer d1f79a0bcbdb42fe89e7812efeae1a86 ({username='slave'} at 127.0.97.1:44623)
I20260929 20:47:49.031370 1156 tablet_copy_service.cc:161] P cac815b9c12e40c6b63d4af82248f298: Beginning new tablet copy session on tablet 9780cc79845d46d780893950ca742c02 from peer d1f79a0bcbdb42fe89e7812efeae1a86 at {username='slave'} at 127.0.97.1:44623: session id = d1f79a0bcbdb42fe89e7812efeae1a86-9780cc79845d46d780893950ca742c02
I20260929 20:47:49.034241 1156 tablet_copy_source_session.cc:205] T 9780cc79845d46d780893950ca742c02 P cac815b9c12e40c6b63d4af82248f298: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 20:47:49.036404 1669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9780cc79845d46d780893950ca742c02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:49.041065 1669 tablet_copy_client.cc:791] T 9780cc79845d46d780893950ca742c02 P d1f79a0bcbdb42fe89e7812efeae1a86: tablet copy: Starting download of 0 data blocks...
I20260929 20:47:49.041380 1669 tablet_copy_client.cc:655] T 9780cc79845d46d780893950ca742c02 P d1f79a0bcbdb42fe89e7812efeae1a86: tablet copy: Starting download of 1 WAL segments...
I20260929 20:47:49.043819 1669 tablet_copy_client.cc:531] T 9780cc79845d46d780893950ca742c02 P d1f79a0bcbdb42fe89e7812efeae1a86: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 20:47:49.046586 1669 tablet_bootstrap.cc:492] T 9780cc79845d46d780893950ca742c02 P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap starting.
I20260929 20:47:49.052721 1669 tablet_bootstrap.cc:492] T 9780cc79845d46d780893950ca742c02 P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 20:47:49.053139 1669 tablet_bootstrap.cc:492] T 9780cc79845d46d780893950ca742c02 P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap complete.
I20260929 20:47:49.053391 1669 ts_tablet_manager.cc:1397] T 9780cc79845d46d780893950ca742c02 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.001s
I20260929 20:47:49.054055 1669 raft_consensus.cc:348] T 9780cc79845d46d780893950ca742c02 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:49.054265 1669 raft_consensus.cc:729] T 9780cc79845d46d780893950ca742c02 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Initialized, Role: FOLLOWER
I20260929 20:47:49.054560 1669 consensus_queue.cc:260] T 9780cc79845d46d780893950ca742c02 P d1f79a0bcbdb42fe89e7812efeae1a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:49.055123 1669 ts_tablet_manager.cc:1428] T 9780cc79845d46d780893950ca742c02 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260929 20:47:49.056810 1156 tablet_copy_service.cc:342] P cac815b9c12e40c6b63d4af82248f298: Request end of tablet copy session d1f79a0bcbdb42fe89e7812efeae1a86-9780cc79845d46d780893950ca742c02 received from {username='slave'} at 127.0.97.1:44623
I20260929 20:47:49.057010 1156 tablet_copy_service.cc:433] P cac815b9c12e40c6b63d4af82248f298: ending tablet copy session d1f79a0bcbdb42fe89e7812efeae1a86-9780cc79845d46d780893950ca742c02 on tablet 9780cc79845d46d780893950ca742c02 with peer d1f79a0bcbdb42fe89e7812efeae1a86
I20260929 20:47:49.176323 1669 ts_tablet_manager.cc:927] T 5e20f360f9b74f3f929b7acfdf82671f P d1f79a0bcbdb42fe89e7812efeae1a86: Initiating tablet copy from peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:49.176903 1669 tablet_copy_client.cc:316] T 5e20f360f9b74f3f929b7acfdf82671f P d1f79a0bcbdb42fe89e7812efeae1a86: tablet copy: Beginning tablet copy session from remote peer at address 127.0.97.3:32835
I20260929 20:47:49.177752 1298 tablet_copy_service.cc:140] P 1cb32e9690a546149d11339d371c832f: Received BeginTabletCopySession request for tablet 5e20f360f9b74f3f929b7acfdf82671f from peer d1f79a0bcbdb42fe89e7812efeae1a86 ({username='slave'} at 127.0.97.1:55977)
I20260929 20:47:49.177976 1298 tablet_copy_service.cc:161] P 1cb32e9690a546149d11339d371c832f: Beginning new tablet copy session on tablet 5e20f360f9b74f3f929b7acfdf82671f from peer d1f79a0bcbdb42fe89e7812efeae1a86 at {username='slave'} at 127.0.97.1:55977: session id = d1f79a0bcbdb42fe89e7812efeae1a86-5e20f360f9b74f3f929b7acfdf82671f
I20260929 20:47:49.180927 1298 tablet_copy_source_session.cc:205] T 5e20f360f9b74f3f929b7acfdf82671f P 1cb32e9690a546149d11339d371c832f: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 20:47:49.182631 1669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e20f360f9b74f3f929b7acfdf82671f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:49.187350 1669 tablet_copy_client.cc:791] T 5e20f360f9b74f3f929b7acfdf82671f P d1f79a0bcbdb42fe89e7812efeae1a86: tablet copy: Starting download of 0 data blocks...
I20260929 20:47:49.187637 1669 tablet_copy_client.cc:655] T 5e20f360f9b74f3f929b7acfdf82671f P d1f79a0bcbdb42fe89e7812efeae1a86: tablet copy: Starting download of 1 WAL segments...
I20260929 20:47:49.189819 1669 tablet_copy_client.cc:531] T 5e20f360f9b74f3f929b7acfdf82671f P d1f79a0bcbdb42fe89e7812efeae1a86: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 20:47:49.192235 1669 tablet_bootstrap.cc:492] T 5e20f360f9b74f3f929b7acfdf82671f P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap starting.
I20260929 20:47:49.196781 1669 tablet_bootstrap.cc:492] T 5e20f360f9b74f3f929b7acfdf82671f P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 20:47:49.197161 1669 tablet_bootstrap.cc:492] T 5e20f360f9b74f3f929b7acfdf82671f P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap complete.
I20260929 20:47:49.197395 1669 ts_tablet_manager.cc:1397] T 5e20f360f9b74f3f929b7acfdf82671f P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent bootstrapping tablet: real 0.005s user 0.006s sys 0.001s
I20260929 20:47:49.198020 1669 raft_consensus.cc:348] T 5e20f360f9b74f3f929b7acfdf82671f P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:49.198190 1669 raft_consensus.cc:729] T 5e20f360f9b74f3f929b7acfdf82671f P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Initialized, Role: FOLLOWER
I20260929 20:47:49.198473 1669 consensus_queue.cc:260] T 5e20f360f9b74f3f929b7acfdf82671f P d1f79a0bcbdb42fe89e7812efeae1a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } }
I20260929 20:47:49.199200 1669 ts_tablet_manager.cc:1428] T 5e20f360f9b74f3f929b7acfdf82671f P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 20:47:49.201004 1298 tablet_copy_service.cc:342] P 1cb32e9690a546149d11339d371c832f: Request end of tablet copy session d1f79a0bcbdb42fe89e7812efeae1a86-5e20f360f9b74f3f929b7acfdf82671f received from {username='slave'} at 127.0.97.1:55977
I20260929 20:47:49.201323 1298 tablet_copy_service.cc:433] P 1cb32e9690a546149d11339d371c832f: ending tablet copy session d1f79a0bcbdb42fe89e7812efeae1a86-5e20f360f9b74f3f929b7acfdf82671f on tablet 5e20f360f9b74f3f929b7acfdf82671f with peer d1f79a0bcbdb42fe89e7812efeae1a86
I20260929 20:47:49.626430 1669 ts_tablet_manager.cc:927] T c2861c34ac464b9993b200df3fc201a1 P d1f79a0bcbdb42fe89e7812efeae1a86: Initiating tablet copy from peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:49.627264 1669 tablet_copy_client.cc:316] T c2861c34ac464b9993b200df3fc201a1 P d1f79a0bcbdb42fe89e7812efeae1a86: tablet copy: Beginning tablet copy session from remote peer at address 127.0.97.3:32835
I20260929 20:47:49.628038 1298 tablet_copy_service.cc:140] P 1cb32e9690a546149d11339d371c832f: Received BeginTabletCopySession request for tablet c2861c34ac464b9993b200df3fc201a1 from peer d1f79a0bcbdb42fe89e7812efeae1a86 ({username='slave'} at 127.0.97.1:55977)
I20260929 20:47:49.628239 1298 tablet_copy_service.cc:161] P 1cb32e9690a546149d11339d371c832f: Beginning new tablet copy session on tablet c2861c34ac464b9993b200df3fc201a1 from peer d1f79a0bcbdb42fe89e7812efeae1a86 at {username='slave'} at 127.0.97.1:55977: session id = d1f79a0bcbdb42fe89e7812efeae1a86-c2861c34ac464b9993b200df3fc201a1
I20260929 20:47:49.630555 1298 tablet_copy_source_session.cc:205] T c2861c34ac464b9993b200df3fc201a1 P 1cb32e9690a546149d11339d371c832f: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 20:47:49.631680 1669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2861c34ac464b9993b200df3fc201a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 20:47:49.635926 1669 tablet_copy_client.cc:791] T c2861c34ac464b9993b200df3fc201a1 P d1f79a0bcbdb42fe89e7812efeae1a86: tablet copy: Starting download of 0 data blocks...
I20260929 20:47:49.636186 1669 tablet_copy_client.cc:655] T c2861c34ac464b9993b200df3fc201a1 P d1f79a0bcbdb42fe89e7812efeae1a86: tablet copy: Starting download of 1 WAL segments...
I20260929 20:47:49.637806 1669 tablet_copy_client.cc:531] T c2861c34ac464b9993b200df3fc201a1 P d1f79a0bcbdb42fe89e7812efeae1a86: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 20:47:49.640033 1669 tablet_bootstrap.cc:492] T c2861c34ac464b9993b200df3fc201a1 P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap starting.
I20260929 20:47:49.645887 1669 tablet_bootstrap.cc:492] T c2861c34ac464b9993b200df3fc201a1 P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 20:47:49.646307 1669 tablet_bootstrap.cc:492] T c2861c34ac464b9993b200df3fc201a1 P d1f79a0bcbdb42fe89e7812efeae1a86: Bootstrap complete.
I20260929 20:47:49.646552 1669 ts_tablet_manager.cc:1397] T c2861c34ac464b9993b200df3fc201a1 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 20:47:49.647169 1669 raft_consensus.cc:348] T c2861c34ac464b9993b200df3fc201a1 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:49.647344 1669 raft_consensus.cc:729] T c2861c34ac464b9993b200df3fc201a1 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f79a0bcbdb42fe89e7812efeae1a86, State: Initialized, Role: FOLLOWER
I20260929 20:47:49.647581 1669 consensus_queue.cc:260] T c2861c34ac464b9993b200df3fc201a1 P d1f79a0bcbdb42fe89e7812efeae1a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:49.648309 388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
/tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.62:46175
--webserver_interface=127.0.97.62
--webserver_port=40179
--builtin_ntp_servers=127.0.97.20:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260929 20:47:49.648473 1669 ts_tablet_manager.cc:1428] T c2861c34ac464b9993b200df3fc201a1 P d1f79a0bcbdb42fe89e7812efeae1a86: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
I20260929 20:47:49.649430 1298 tablet_copy_service.cc:342] P 1cb32e9690a546149d11339d371c832f: Request end of tablet copy session d1f79a0bcbdb42fe89e7812efeae1a86-c2861c34ac464b9993b200df3fc201a1 received from {username='slave'} at 127.0.97.1:55977
I20260929 20:47:49.649569 1298 tablet_copy_service.cc:433] P 1cb32e9690a546149d11339d371c832f: ending tablet copy session d1f79a0bcbdb42fe89e7812efeae1a86-c2861c34ac464b9993b200df3fc201a1 on tablet c2861c34ac464b9993b200df3fc201a1 with peer d1f79a0bcbdb42fe89e7812efeae1a86
I20260929 20:47:49.744280 1679 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:49.744556 1679 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:49.745278 1679 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:47:49.746368 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:47:49.746570 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:47:49.746657 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:47:49.746932 1679 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:47:50.005602 1679 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:50.005815 1679 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:50.006577 1679 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:47:50.007571 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:47:50.007781 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:47:50.007922 924 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:47:50.008210 1679 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260929 20:47:50.008174 1684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 20:47:50.008616 1684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 20:47:50.008764 1684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 20:47:50.020304 1684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 20:47:50.020457 1684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 20:47:50.020535 1684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 20:47:50.020609 1684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 20:47:50.033660 1684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.97.20:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.62:46175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.97.62
--webserver_port=40179
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 19:57:29 UTC on ccycloud.quasar-xxilbq.root.comops.site
build id 4681269
ASAN enabled
I20260929 20:47:50.035108 1684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 20:47:50.036887 1684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 20:47:50.050171 1689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:50.050189 1692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:50.050212 1690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 20:47:50.051106 1684 server_base.cc:1034] running on GCE node
I20260929 20:47:50.052057 1684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 20:47:50.053751 1684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 20:47:50.055007 1684 hybrid_clock.cc:648] HybridClock initialized: now 1790714870054923 us; error 62 us; skew 500 ppm
I20260929 20:47:50.055414 1684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 20:47:50.058852 1684 webserver.cc:460] Webserver started at http://127.0.97.62:40179/ using document root <none> and password file <none>
I20260929 20:47:50.059621 1684 fs_manager.cc:359] Metadata directory not provided
I20260929 20:47:50.059787 1684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 20:47:50.065587 1684 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 20:47:50.067219 990 raft_consensus.cc:1203] T 9780cc79845d46d780893950ca742c02 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20260929 20:47:50.071017 1701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 20:47:50.072723 1684 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.001s
I20260929 20:47:50.072889 1684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data,/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8"
format_stamp: "Formatted at 2026-09-29 20:47:41 on dist-test-slave-j0bz"
I20260929 20:47:50.073882 1684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 20:47:50.109143 1684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 20:47:50.110024 1684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 20:47:50.110405 1684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 20:47:50.128147 990 raft_consensus.cc:1203] T 5e20f360f9b74f3f929b7acfdf82671f P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20260929 20:47:50.137037 1684 rpc_server.cc:307] RPC server started. Bound to: 127.0.97.62:46175
I20260929 20:47:50.137082 1757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.97.62:46175 every 8 connection(s)
I20260929 20:47:50.138629 1684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
I20260929 20:47:50.141382 388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu as pid 1684
I20260929 20:47:50.146617 1758 sys_catalog.cc:263] Verifying existing consensus state
I20260929 20:47:50.151741 1758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Bootstrap starting.
I20260929 20:47:50.163199 1758 log.cc:824] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Log is configured to *not* fsync() on all Append() calls
I20260929 20:47:50.190831 1758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=33 overwritten=0 applied=33 ignored=0} inserts{seen=24 ignored=0} mutations{seen=19 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 20:47:50.191370 1758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Bootstrap complete.
I20260929 20:47:50.199311 1758 raft_consensus.cc:348] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:50.200234 1758 raft_consensus.cc:729] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32aaf3821ff4ad09bcbfe11ff2e03c8, State: Initialized, Role: FOLLOWER
I20260929 20:47:50.200959 1758 consensus_queue.cc:260] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 33, Last appended: 3.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:50.201828 1762 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:50.202265 1762 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:50.202950 1758 sys_catalog.cc:564] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 20:47:50.229775 1777 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:47:50.230041 1777 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: loading cluster ID for follower catalog manager: success
I20260929 20:47:50.233677 1777 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: acquiring CA information for follower catalog manager: success
I20260929 20:47:50.235962 1777 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 20:47:50.228878 1684 master_runner.cc:186] Error getting master registration for 127.0.97.61:34103: Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111),
I20260929 20:47:50.239055 1684 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:47:50.340444 1781 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cc99fb1b059842088a50ba9c56fda6c1)
I20260929 20:47:50.340744 1781 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:50.341486 1781 leader_election.cc:288] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103)
W20260929 20:47:50.345082 1484 leader_election.cc:334] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
I20260929 20:47:50.350744 1730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfbad3ef4593415191fa0d993866d0ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 39 } } ignore_live_leader: false dest_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" is_pre_election: true
I20260929 20:47:50.351514 1730 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfbad3ef4593415191fa0d993866d0ea in term 3.
I20260929 20:47:50.352401 1485 leader_election.cc:302] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea; no voters: cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:47:50.352787 1781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260929 20:47:50.352916 1781 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader cc99fb1b059842088a50ba9c56fda6c1)
I20260929 20:47:50.353047 1781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 4 FOLLOWER]: Advancing to term 5
I20260929 20:47:50.355011 1781 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:50.355674 1781 leader_election.cc:288] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 election: Requested vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103)
I20260929 20:47:50.356187 1730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfbad3ef4593415191fa0d993866d0ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 39 } } ignore_live_leader: false dest_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8"
I20260929 20:47:50.356410 1730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 3 FOLLOWER]: Advancing to term 5
W20260929 20:47:50.357285 1484 leader_election.cc:334] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
I20260929 20:47:50.361213 1730 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfbad3ef4593415191fa0d993866d0ea in term 5.
I20260929 20:47:50.361734 1485 leader_election.cc:302] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea; no voters: cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:47:50.362103 1781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 5 FOLLOWER]: Leader election won for term 5
I20260929 20:47:50.362628 1781 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 5 LEADER]: Becoming Leader. State: Replica: bfbad3ef4593415191fa0d993866d0ea, State: Running, Role: LEADER
I20260929 20:47:50.363142 1781 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 4.39, Last appended by leader: 39, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:50.365077 1785 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfbad3ef4593415191fa0d993866d0ea. Latest consensus state: current_term: 5 leader_uuid: "bfbad3ef4593415191fa0d993866d0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:50.365515 1785 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: This master's current role is: LEADER
I20260929 20:47:50.366225 1787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 20:47:50.368937 1787 catalog_manager.cc:669] Loaded metadata for table default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]
I20260929 20:47:50.369414 1787 catalog_manager.cc:669] Loaded metadata for table default.table_5b5c251fd82449869826c01b42a62f31 [id=3842db9126434978a3631262acd85d32]
I20260929 20:47:50.369771 1787 catalog_manager.cc:669] Loaded metadata for table default.table_0b803212f31d45419130e1da10d1a620 [id=796a2c5810ac4950a0f9499208fad0e9]
I20260929 20:47:50.370204 1787 catalog_manager.cc:669] Loaded metadata for table default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f]
I20260929 20:47:50.370616 1787 catalog_manager.cc:669] Loaded metadata for table default.table_556595b54fef48f99c72c48e730495e9 [id=b17facaee1924496b1beaf3d695c11ab]
I20260929 20:47:50.371057 1787 catalog_manager.cc:669] Loaded metadata for table default.table_f02a75c1e46d472d90334b4340d99e7b [id=dd1c5cbd5c6c40fc9fb5cc1dbe7026b0]
I20260929 20:47:50.371505 1787 catalog_manager.cc:669] Loaded metadata for table default.table_a5aa493381b543419904f1429ffc19f2 [id=e7f4bcce9df4454a81f6e98a5b2e69e5]
I20260929 20:47:50.371948 1787 catalog_manager.cc:669] Loaded metadata for table default.table_952135bfbb564f519938b6b206b1d463 [id=f04d505b1ef9453780f36b454bc22f46]
I20260929 20:47:50.372396 1787 catalog_manager.cc:669] Loaded metadata for table default.table_47f3dc18fe564237bda167352f1afa08 [id=f825b412c46842ec967df7bb7d8f9474]
I20260929 20:47:50.374836 1787 tablet_loader.cc:96] loaded metadata for tablet 061c178e50924b51a92542bea703b24c (table default.table_556595b54fef48f99c72c48e730495e9 [id=b17facaee1924496b1beaf3d695c11ab])
I20260929 20:47:50.376091 1787 tablet_loader.cc:96] loaded metadata for tablet 5e20f360f9b74f3f929b7acfdf82671f (table default.table_47f3dc18fe564237bda167352f1afa08 [id=f825b412c46842ec967df7bb7d8f9474])
I20260929 20:47:50.376430 1787 tablet_loader.cc:96] loaded metadata for tablet 6820644bb2a745679d959d8ac6854014 (table default.table_f02a75c1e46d472d90334b4340d99e7b [id=dd1c5cbd5c6c40fc9fb5cc1dbe7026b0])
I20260929 20:47:50.376775 1787 tablet_loader.cc:96] loaded metadata for tablet 9201e11b19f84f0db4e2b132a1b60ff8 (table default.table_0b803212f31d45419130e1da10d1a620 [id=796a2c5810ac4950a0f9499208fad0e9])
I20260929 20:47:50.377080 1787 tablet_loader.cc:96] loaded metadata for tablet 95476e3b89fe467ea3f151400dcc4f76 (table default.table_952135bfbb564f519938b6b206b1d463 [id=f04d505b1ef9453780f36b454bc22f46])
I20260929 20:47:50.377408 1787 tablet_loader.cc:96] loaded metadata for tablet 9780cc79845d46d780893950ca742c02 (table default.table_a5aa493381b543419904f1429ffc19f2 [id=e7f4bcce9df4454a81f6e98a5b2e69e5])
I20260929 20:47:50.377835 1787 tablet_loader.cc:96] loaded metadata for tablet b6df369f96d54b33997ee5d4b4860df9 (table default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a])
I20260929 20:47:50.378134 1787 tablet_loader.cc:96] loaded metadata for tablet c2861c34ac464b9993b200df3fc201a1 (table default.table_5b5c251fd82449869826c01b42a62f31 [id=3842db9126434978a3631262acd85d32])
I20260929 20:47:50.378425 1787 tablet_loader.cc:96] loaded metadata for tablet c4a57da4e5c24803b7aa3f61c7191e38 (table default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f])
I20260929 20:47:50.378865 1787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 20:47:50.380137 1787 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:47:50.380245 1787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 20:47:50.381693 1787 catalog_manager.cc:1504] Loading token signing keys...
I20260929 20:47:50.382629 1787 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: Loaded TSK: 0
I20260929 20:47:50.383495 1787 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 20:47:50.384404 1787 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260929 20:47:50.385298 1787 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260929 20:47:50.431255 1037 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:47:50.438126 1720 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" instance_seqno: 1790714863424856) as {username='slave'} at 127.0.97.1:41837; Asking this server to re-register.
I20260929 20:47:50.439527 1037 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:50.441968 1720 ts_manager.cc:194] Registered new tserver with Master: d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657)
I20260929 20:47:50.458474 1500 master_service.cc:946] ReplaceTablet: received request to replace tablet 95476e3b89fe467ea3f151400dcc4f76 from {username='slave'} at 127.0.0.1:36424
I20260929 20:47:50.461143 1493 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36424:
table { table_name: "default.Table_0b803212f31d45419130e1da10d1a620" } new_table_name: "default.Table_71dc24415d9046608c47e70ce90636c4" modify_external_catalogs: true
I20260929 20:47:50.462077 1497 master_service.cc:946] ReplaceTablet: received request to replace tablet 9201e11b19f84f0db4e2b132a1b60ff8 from {username='slave'} at 127.0.0.1:36424
I20260929 20:47:50.463577 1496 master_service.cc:946] ReplaceTablet: received request to replace tablet 95476e3b89fe467ea3f151400dcc4f76 from {username='slave'} at 127.0.0.1:36424
I20260929 20:47:50.464483 1495 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36424:
name: "default.Table_e1a3cd10e94d42c598e9c071db2144d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:50.465376 1494 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36424:
name: "default.Table_a988b1827f9e482086b9b9ad0663c2b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:50.466027 1492 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36424:
table { table_name: "default.Table_a5aa493381b543419904f1429ffc19f2" } new_table_name: "default.Table_4eeaf34187a44f96b98786696ca454c5" modify_external_catalogs: true
I20260929 20:47:50.466519 1491 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:36424:
table { table_name: "default.Table_d9ce7635a33b4f6ea325f294cc7db677" } new_table_name: "default.Table_d9ff14af9acd4cc4a2a2a7bc7d22d17f" modify_external_catalogs: true
I20260929 20:47:50.470937 971 tablet_service.cc:1514] Processing DeleteTablet for tablet 53c949bb99444778a9f15a781596e986 with delete_type TABLET_DATA_DELETED (replaced by tablet 9780cc79845d46d780893950ca742c02) from {username='slave'} at 127.0.0.1:47698
I20260929 20:47:50.471875 1793 tablet_replica.cc:330] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86: stopping tablet replica
I20260929 20:47:50.472169 1793 raft_consensus.cc:2229] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 LEADER]: Raft consensus shutting down.
I20260929 20:47:50.472851 1793 pending_rounds.cc:62] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86: Trying to abort 1 pending ops.
I20260929 20:47:50.473013 1793 pending_rounds.cc:68] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334768091484635136 op_type: NO_OP noop_request { }
I20260929 20:47:50.473332 1793 raft_consensus.cc:2866] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260929 20:47:50.473309 1498 master_service.cc:946] ReplaceTablet: received request to replace tablet 9201e11b19f84f0db4e2b132a1b60ff8 from {username='slave'} at 127.0.0.1:36424
I20260929 20:47:50.473524 1793 raft_consensus.cc:2258] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 20:47:50.473645 1499 master_service.cc:946] ReplaceTablet: received request to replace tablet 9201e11b19f84f0db4e2b132a1b60ff8 from {username='slave'} at 127.0.0.1:36424
I20260929 20:47:50.474401 1793 ts_tablet_manager.cc:1905] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 20:47:50.483623 1793 ts_tablet_manager.cc:1918] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260929 20:47:50.484077 1793 log.cc:1196] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86: Deleting WAL directory at /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/ts-0/wal/wals/53c949bb99444778a9f15a781596e986
F20260929 20:47:50.484551 1559 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_4cf3799777554d47b5dbb898a6d368b9
I20260929 20:47:50.484689 1793 ts_tablet_manager.cc:1939] T 53c949bb99444778a9f15a781596e986 P d1f79a0bcbdb42fe89e7812efeae1a86: Deleting consensus metadata
*** Check failure stack trace: ***
I20260929 20:47:50.485972 1484 catalog_manager.cc:4841] TS d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657): tablet 53c949bb99444778a9f15a781596e986 (table default.table_a5aa493381b543419904f1429ffc19f2 [id=e7f4bcce9df4454a81f6e98a5b2e69e5]) successfully deleted
@ 0x7f0a14cf99ad google::LogMessage::Fail() at ??:0
@ 0x7f0a14cfc88a google::LogMessage::SendToLog() at ??:0
@ 0x7f0a14cf94de google::LogMessage::Flush() at ??:0
@ 0x7f0a14cf96d1 google::LogMessage::~LogMessage() at ??:0
I20260929 20:47:50.499308 1730 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 5 FOLLOWER]: Refusing update from remote peer bfbad3ef4593415191fa0d993866d0ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 33. Preceding OpId from leader: term: 5 index: 41. (index mismatch)
W20260929 20:47:50.500159 1484 consensus_peers.cc:487] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea -> Peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Couldn't send request to peer cc99fb1b059842088a50ba9c56fda6c1. Status: Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 20:47:50.500432 1785 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 33, Time since last communication: 0.000s
@ 0x7f0a2a897ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f0a2a896901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f0a2a89d66c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f0a2a89d4e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0a17923eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0a1abfd6db start_thread at ??:0
@ 0x7f0a13b9361f clone at ??:0
I20260929 20:47:50.556648 1797 mvcc.cc:205] Tried to move back new op lower bound from 7334768095080194048 to 7334768094387544064. Current Snapshot: MvccSnapshot[applied={T|T < 7334768095080194048}]
I20260929 20:47:50.556890 1762 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfbad3ef4593415191fa0d993866d0ea. Latest consensus state: current_term: 5 leader_uuid: "bfbad3ef4593415191fa0d993866d0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:50.557261 1762 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:50.557694 1801 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "bfbad3ef4593415191fa0d993866d0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:50.558120 1801 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:50.578338 1182 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:47:50.581283 1720 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" instance_seqno: 1790714863999745) as {username='slave'} at 127.0.97.2:54163; Asking this server to re-register.
I20260929 20:47:50.582186 1182 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:50.584035 1720 ts_manager.cc:194] Registered new tserver with Master: cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:50.638190 1324 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:47:50.641152 1720 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "1cb32e9690a546149d11339d371c832f" instance_seqno: 1790714864581790) as {username='slave'} at 127.0.97.3:36419; Asking this server to re-register.
I20260929 20:47:50.642184 1324 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:50.644389 1719 ts_manager.cc:194] Registered new tserver with Master: 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:47:50.751484 1071 connection.cc:535] client connection to 127.0.97.60:45599 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 20:47:50.751469 1213 connection.cc:535] client connection to 127.0.97.60:45599 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 20:47:50.751772 1706 connection.cc:535] server connection from 127.0.0.1:36380 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 20:47:50.752367 1333 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.97.60:45599)
W20260929 20:47:50.753012 1333 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.97.60:45599)
W20260929 20:47:50.753793 1333 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.97.60:45599)
W20260929 20:47:50.754369 1333 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.97.60:45599)
I20260929 20:47:50.765823 990 raft_consensus.cc:1203] T c2861c34ac464b9993b200df3fc201a1 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20260929 20:47:50.766400 1036 heartbeater.cc:503] Master 127.0.97.60:45599 was elected leader, sending a full tablet report...
I20260929 20:47:51.003396 1801 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader bfbad3ef4593415191fa0d993866d0ea)
I20260929 20:47:51.003647 1801 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:51.005635 1801 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:47:51.006850 1706 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:47:51.008566 1702 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
W20260929 20:47:51.008858 1706 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:47:51.009032 1706 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8; no voters: bfbad3ef4593415191fa0d993866d0ea, cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:47:51.009400 1801 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260929 20:47:51.722416 1820 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader bfbad3ef4593415191fa0d993866d0ea)
I20260929 20:47:51.722589 1820 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:51.723214 1820 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:47:51.725528 1702 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
W20260929 20:47:51.725729 1706 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:47:51.725847 1706 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8; no voters: bfbad3ef4593415191fa0d993866d0ea, cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:47:51.726128 1820 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260929 20:47:52.399351 1828 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader bfbad3ef4593415191fa0d993866d0ea)
I20260929 20:47:52.399623 1828 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:52.400259 1828 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:47:52.402657 1706 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:47:52.402879 1702 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
I20260929 20:47:52.403018 1702 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8; no voters: bfbad3ef4593415191fa0d993866d0ea, cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:47:52.403331 1828 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260929 20:47:53.046675 1788 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:53.046955 1788 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:53.047801 1788 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:47:53.048869 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:47:53.049104 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:47:53.049286 925 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:47:53.049705 1788 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:47:53.065606 388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
/tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.60:45599
--webserver_interface=127.0.97.60
--webserver_port=43799
--builtin_ntp_servers=127.0.97.20:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260929 20:47:53.219403 1788 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:53.219650 1788 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:53.220476 1788 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:47:53.221421 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:47:53.221648 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:47:53.221825 925 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:47:53.222095 1788 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260929 20:47:53.420974 1837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 20:47:53.421370 1837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 20:47:53.421536 1837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 20:47:53.429476 1837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 20:47:53.429629 1837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 20:47:53.429728 1837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 20:47:53.429844 1837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 20:47:53.439625 1837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.97.20:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.60:45599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.97.60
--webserver_port=43799
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 19:57:29 UTC on ccycloud.quasar-xxilbq.root.comops.site
build id 4681269
ASAN enabled
I20260929 20:47:53.441143 1837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 20:47:53.443089 1837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 20:47:53.455813 1846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:53.457269 1845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:53.457504 1848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 20:47:53.458269 1837 server_base.cc:1034] running on GCE node
I20260929 20:47:53.459251 1837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 20:47:53.460783 1837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 20:47:53.462033 1837 hybrid_clock.cc:648] HybridClock initialized: now 1790714873461967 us; error 46 us; skew 500 ppm
I20260929 20:47:53.462409 1837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 20:47:53.464874 1837 webserver.cc:460] Webserver started at http://127.0.97.60:43799/ using document root <none> and password file <none>
I20260929 20:47:53.465629 1837 fs_manager.cc:359] Metadata directory not provided
I20260929 20:47:53.465811 1837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 20:47:53.471094 1837 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260929 20:47:53.474251 1854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 20:47:53.475836 1837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260929 20:47:53.476027 1837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data,/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/wal
uuid: "bfbad3ef4593415191fa0d993866d0ea"
format_stamp: "Formatted at 2026-09-29 20:47:42 on dist-test-slave-j0bz"
I20260929 20:47:53.477089 1837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 20:47:53.499758 1837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 20:47:53.500732 1837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 20:47:53.501053 1837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 20:47:53.513429 1887 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader bfbad3ef4593415191fa0d993866d0ea)
I20260929 20:47:53.513739 1887 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:53.514724 1887 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:47:53.517951 1702 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
W20260929 20:47:53.518590 1706 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:47:53.518766 1706 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8; no voters: bfbad3ef4593415191fa0d993866d0ea, cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:47:53.519079 1887 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260929 20:47:53.523255 1837 rpc_server.cc:307] RPC server started. Bound to: 127.0.97.60:45599
I20260929 20:47:53.523275 1909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.97.60:45599 every 8 connection(s)
I20260929 20:47:53.524907 1837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-2/data/info.pb
I20260929 20:47:53.525751 388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu as pid 1837
I20260929 20:47:53.534176 1910 sys_catalog.cc:263] Verifying existing consensus state
I20260929 20:47:53.539194 1910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: Bootstrap starting.
I20260929 20:47:53.551209 1910 log.cc:824] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: Log is configured to *not* fsync() on all Append() calls
I20260929 20:47:53.581735 1910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=42 overwritten=0 applied=39 ignored=0} inserts{seen=27 ignored=0} mutations{seen=25 ignored=0} orphaned_commits=0. Pending: 3 replicates
I20260929 20:47:53.582387 1910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: Bootstrap complete.
I20260929 20:47:53.590065 1910 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 5 FOLLOWER]: Replica starting. Triggering 3 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:53.593686 1910 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfbad3ef4593415191fa0d993866d0ea, State: Initialized, Role: FOLLOWER
I20260929 20:47:53.594452 1910 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 39, Last appended: 5.42, Last appended by leader: 42, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:53.595361 1916 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:53.595821 1916 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:53.596920 1910 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: configured and running, proceeding with master startup.
I20260929 20:47:53.633038 1930 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:47:53.633313 1930 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: loading cluster ID for follower catalog manager: success
I20260929 20:47:53.638850 1930 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: acquiring CA information for follower catalog manager: success
I20260929 20:47:53.642352 1930 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 20:47:53.640409 1837 master_runner.cc:186] Error getting master registration for 127.0.97.61:34103: Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111),
I20260929 20:47:53.650578 1837 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:47:53.767014 1181 heartbeater.cc:347] Connected to a master server at 127.0.97.60:45599
I20260929 20:47:53.770535 1870 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" instance_seqno: 1790714863999745) as {username='slave'} at 127.0.97.2:54617; Asking this server to re-register.
I20260929 20:47:53.771432 1181 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:53.773561 1870 ts_manager.cc:194] Registered new tserver with Master: cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:53.786793 1036 heartbeater.cc:347] Connected to a master server at 127.0.97.60:45599
I20260929 20:47:53.786866 1323 heartbeater.cc:347] Connected to a master server at 127.0.97.60:45599
I20260929 20:47:53.787034 1036 heartbeater.cc:503] Master 127.0.97.60:45599 was elected leader, sending a full tablet report...
I20260929 20:47:53.789111 1871 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "1cb32e9690a546149d11339d371c832f" instance_seqno: 1790714864581790) as {username='slave'} at 127.0.97.3:49891; Asking this server to re-register.
I20260929 20:47:53.789232 1870 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" instance_seqno: 1790714863424856) as {username='slave'} at 127.0.97.1:34617; Asking this server to re-register.
I20260929 20:47:53.789726 1323 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:53.791316 1036 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:53.791303 1870 ts_manager.cc:194] Registered new tserver with Master: 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:53.792977 1870 ts_manager.cc:194] Registered new tserver with Master: d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657)
I20260929 20:47:54.204471 1938 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:54.204846 1938 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:54.206887 1938 leader_election.cc:288] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103)
I20260929 20:47:54.207465 1730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfbad3ef4593415191fa0d993866d0ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 42 } } ignore_live_leader: false dest_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" is_pre_election: true
I20260929 20:47:54.207743 1730 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfbad3ef4593415191fa0d993866d0ea in term 5.
W20260929 20:47:54.207957 1855 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
I20260929 20:47:54.208480 1856 leader_election.cc:302] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea; no voters:
I20260929 20:47:54.209017 1938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260929 20:47:54.209177 1938 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:47:54.209322 1938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 5 FOLLOWER]: Advancing to term 6
W20260929 20:47:54.209659 1855 leader_election.cc:334] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
I20260929 20:47:54.213685 1938 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:54.214424 1938 leader_election.cc:288] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 6 election: Requested vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103)
I20260929 20:47:54.214963 1730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfbad3ef4593415191fa0d993866d0ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 42 } } ignore_live_leader: false dest_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8"
I20260929 20:47:54.215211 1730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 5 FOLLOWER]: Advancing to term 6
W20260929 20:47:54.216130 1855 leader_election.cc:334] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
I20260929 20:47:54.217491 1730 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfbad3ef4593415191fa0d993866d0ea in term 6.
I20260929 20:47:54.217963 1856 leader_election.cc:302] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea; no voters: cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:47:54.218333 1938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 6 FOLLOWER]: Leader election won for term 6
I20260929 20:47:54.218729 1938 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [term 6 LEADER]: Becoming Leader. State: Replica: bfbad3ef4593415191fa0d993866d0ea, State: Running, Role: LEADER
I20260929 20:47:54.219211 1938 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 5.42, Last appended by leader: 42, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:54.222093 1941 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfbad3ef4593415191fa0d993866d0ea. Latest consensus state: current_term: 6 leader_uuid: "bfbad3ef4593415191fa0d993866d0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:54.222476 1941 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: This master's current role is: LEADER
I20260929 20:47:54.743826 1730 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Refusing update from remote peer bfbad3ef4593415191fa0d993866d0ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 42. Preceding OpId from leader: term: 6 index: 43. (index mismatch)
I20260929 20:47:54.744707 1947 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 43, Last known committed idx: 39, Time since last communication: 0.000s
I20260929 20:47:54.747066 1948 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfbad3ef4593415191fa0d993866d0ea. Latest consensus state: current_term: 6 leader_uuid: "bfbad3ef4593415191fa0d993866d0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:54.747395 1948 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:54.753204 1947 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "bfbad3ef4593415191fa0d993866d0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:54.753579 1947 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: This master's current role is: LEADER
I20260929 20:47:54.754822 1950 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "bfbad3ef4593415191fa0d993866d0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:54.755209 1950 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea [sys.catalog]: This master's current role is: LEADER
I20260929 20:47:54.756949 1948 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "bfbad3ef4593415191fa0d993866d0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:54.757171 1948 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 20:47:54.757408 1855 consensus_peers.cc:487] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea -> Peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Couldn't send request to peer cc99fb1b059842088a50ba9c56fda6c1. Status: Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 20:47:54.756871 1960 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "bfbad3ef4593415191fa0d993866d0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:54.757568 1960 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:54.868386 1943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 20:47:54.871938 1943 catalog_manager.cc:669] Loaded metadata for table default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]
I20260929 20:47:54.872611 1943 catalog_manager.cc:669] Loaded metadata for table default.table_5b5c251fd82449869826c01b42a62f31 [id=3842db9126434978a3631262acd85d32]
I20260929 20:47:54.873013 1943 catalog_manager.cc:669] Loaded metadata for table default.table_0b803212f31d45419130e1da10d1a620 [id=796a2c5810ac4950a0f9499208fad0e9]
I20260929 20:47:54.873417 1943 catalog_manager.cc:669] Loaded metadata for table default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f]
I20260929 20:47:54.873775 1943 catalog_manager.cc:669] Loaded metadata for table default.table_556595b54fef48f99c72c48e730495e9 [id=b17facaee1924496b1beaf3d695c11ab]
I20260929 20:47:54.874163 1943 catalog_manager.cc:669] Loaded metadata for table default.table_f02a75c1e46d472d90334b4340d99e7b [id=dd1c5cbd5c6c40fc9fb5cc1dbe7026b0]
I20260929 20:47:54.874511 1943 catalog_manager.cc:669] Loaded metadata for table default.table_a5aa493381b543419904f1429ffc19f2 [id=e7f4bcce9df4454a81f6e98a5b2e69e5]
I20260929 20:47:54.874837 1943 catalog_manager.cc:669] Loaded metadata for table default.table_952135bfbb564f519938b6b206b1d463 [id=f04d505b1ef9453780f36b454bc22f46]
I20260929 20:47:54.875156 1943 catalog_manager.cc:669] Loaded metadata for table default.table_47f3dc18fe564237bda167352f1afa08 [id=f825b412c46842ec967df7bb7d8f9474]
I20260929 20:47:54.876869 1943 tablet_loader.cc:96] loaded metadata for tablet 061c178e50924b51a92542bea703b24c (table default.table_556595b54fef48f99c72c48e730495e9 [id=b17facaee1924496b1beaf3d695c11ab])
I20260929 20:47:54.877490 1943 tablet_loader.cc:96] loaded metadata for tablet 42279556b25240deaf8a689465368262 (table default.table_0b803212f31d45419130e1da10d1a620 [id=796a2c5810ac4950a0f9499208fad0e9])
I20260929 20:47:54.878085 1943 tablet_loader.cc:96] loaded metadata for tablet 5e20f360f9b74f3f929b7acfdf82671f (table default.table_47f3dc18fe564237bda167352f1afa08 [id=f825b412c46842ec967df7bb7d8f9474])
I20260929 20:47:54.878369 1943 tablet_loader.cc:96] loaded metadata for tablet 6820644bb2a745679d959d8ac6854014 (table default.table_f02a75c1e46d472d90334b4340d99e7b [id=dd1c5cbd5c6c40fc9fb5cc1dbe7026b0])
I20260929 20:47:54.879015 1943 tablet_loader.cc:96] loaded metadata for tablet 9780cc79845d46d780893950ca742c02 (table default.table_a5aa493381b543419904f1429ffc19f2 [id=e7f4bcce9df4454a81f6e98a5b2e69e5])
I20260929 20:47:54.879591 1943 tablet_loader.cc:96] loaded metadata for tablet b6df369f96d54b33997ee5d4b4860df9 (table default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a])
I20260929 20:47:54.879916 1943 tablet_loader.cc:96] loaded metadata for tablet c2861c34ac464b9993b200df3fc201a1 (table default.table_5b5c251fd82449869826c01b42a62f31 [id=3842db9126434978a3631262acd85d32])
I20260929 20:47:54.880259 1943 tablet_loader.cc:96] loaded metadata for tablet c4a57da4e5c24803b7aa3f61c7191e38 (table default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f])
I20260929 20:47:54.880600 1943 tablet_loader.cc:96] loaded metadata for tablet c4c5549f5e28423386e92219593420ec (table default.table_952135bfbb564f519938b6b206b1d463 [id=f04d505b1ef9453780f36b454bc22f46])
I20260929 20:47:54.881155 1943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 20:47:54.882464 1943 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:47:54.882548 1943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 20:47:54.883862 1943 catalog_manager.cc:1504] Loading token signing keys...
I20260929 20:47:54.884859 1943 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bfbad3ef4593415191fa0d993866d0ea: Loaded TSK: 0
I20260929 20:47:54.885728 1943 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 20:47:54.886785 1943 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260929 20:47:54.887975 1943 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260929 20:47:54.976378 1871 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41502:
name: "default.Table_e3c4d53b991348e8af5e335ad5847221"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:54.976771 1867 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41502:
table { table_name: "default.Table_5b5c251fd82449869826c01b42a62f31" } modify_external_catalogs: true
I20260929 20:47:54.976918 1866 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41502:
table { table_name: "default.Table_556595b54fef48f99c72c48e730495e9" } modify_external_catalogs: true
I20260929 20:47:54.977119 1867 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41502:
table { table_name: "default.Table_5b5c251fd82449869826c01b42a62f31" } modify_external_catalogs: true
I20260929 20:47:54.977082 1870 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41502:
name: "default.Table_dae8faea7e2b44909906d91d85898732"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:54.976450 1869 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41502:
name: "default.Table_55d763f6ec744322a43d80efeb51983e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:54.977403 1868 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41502:
table { table_name: "default.Table_f02a75c1e46d472d90334b4340d99e7b" } modify_external_catalogs: true
I20260929 20:47:54.977658 1868 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41502:
table { table_name: "default.Table_f02a75c1e46d472d90334b4340d99e7b" } modify_external_catalogs: true
I20260929 20:47:54.978147 1865 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41502:
name: "default.Table_57d2f2bbf5c3465ba1591d6a2b6fbbc0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:54.978551 1862 master_service.cc:946] ReplaceTablet: received request to replace tablet 95476e3b89fe467ea3f151400dcc4f76 from {username='slave'} at 127.0.0.1:41502
I20260929 20:47:54.977284 1866 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41502:
table { table_name: "default.Table_556595b54fef48f99c72c48e730495e9" } modify_external_catalogs: true
I20260929 20:47:54.979014 1863 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41502:
name: "default.Table_e1a3cd10e94d42c598e9c071db2144d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:54.978196 1864 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41502:
name: "default.Table_be3c404093574a04aea5256b4ecba3bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:47:54.980720 1862 master_service.cc:946] ReplaceTablet: received request to replace tablet 9201e11b19f84f0db4e2b132a1b60ff8 from {username='slave'} at 127.0.0.1:41502
I20260929 20:47:54.981171 1862 master_service.cc:946] ReplaceTablet: received request to replace tablet 9201e11b19f84f0db4e2b132a1b60ff8 from {username='slave'} at 127.0.0.1:41502
I20260929 20:47:54.981586 1862 master_service.cc:946] ReplaceTablet: received request to replace tablet 95476e3b89fe467ea3f151400dcc4f76 from {username='slave'} at 127.0.0.1:41502
I20260929 20:47:54.982003 1862 master_service.cc:946] ReplaceTablet: received request to replace tablet 9201e11b19f84f0db4e2b132a1b60ff8 from {username='slave'} at 127.0.0.1:41502
I20260929 20:47:54.982524 1862 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41502:
name: "default.Table_a988b1827f9e482086b9b9ad0663c2b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260929 20:47:54.984167 1929 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_4cf3799777554d47b5dbb898a6d368b9
*** Check failure stack trace: ***
@ 0x7f91be2829ad google::LogMessage::Fail() at ??:0
@ 0x7f91be28588a google::LogMessage::SendToLog() at ??:0
@ 0x7f91be2824de google::LogMessage::Flush() at ??:0
@ 0x7f91be2826d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f91d3e20ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f91d3e1f901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f91d3e2666c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f91d3e264e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f91c0eaceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f91c41866db start_thread at ??:0
@ 0x7f91bd11c61f clone at ??:0
W20260929 20:47:55.210325 1333 connection.cc:535] client connection to 127.0.97.60:45599 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 20:47:55.211091 1333 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.97.60:45599)
W20260929 20:47:55.211760 1333 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.97.60:45599)
W20260929 20:47:55.211997 1333 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.97.60:45599)
W20260929 20:47:55.212455 1333 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.97.60:45599)
I20260929 20:47:55.242048 1684 master_runner.cc:186] Error getting master registration for 127.0.97.61:34103: Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111),
I20260929 20:47:55.242203 1684 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:47:55.249923 1960 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader bfbad3ef4593415191fa0d993866d0ea)
I20260929 20:47:55.250125 1960 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:55.250816 1960 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:47:55.252907 1702 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
W20260929 20:47:55.253225 1706 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:47:55.253387 1706 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8; no voters: bfbad3ef4593415191fa0d993866d0ea, cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:47:55.253670 1960 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260929 20:47:55.714797 388 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu pid 1684
I20260929 20:47:55.722949 1753 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 20:47:55.838205 388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu with pid 1684
I20260929 20:47:56.595155 388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
/tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.62:46175
--webserver_interface=127.0.97.62
--webserver_port=40179
--builtin_ntp_servers=127.0.97.20:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260929 20:47:56.935946 1987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 20:47:56.936255 1987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 20:47:56.936331 1987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 20:47:56.943838 1987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 20:47:56.943951 1987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 20:47:56.943994 1987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 20:47:56.944044 1987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 20:47:56.954038 1987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.97.20:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.62:46175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.97.62
--webserver_port=40179
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 19:57:29 UTC on ccycloud.quasar-xxilbq.root.comops.site
build id 4681269
ASAN enabled
I20260929 20:47:56.955336 1987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 20:47:56.957129 1987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 20:47:56.969540 1995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:56.969663 1993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:47:56.969663 1992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 20:47:56.970315 1987 server_base.cc:1034] running on GCE node
I20260929 20:47:56.971264 1987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 20:47:56.973093 1987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 20:47:56.974417 1987 hybrid_clock.cc:648] HybridClock initialized: now 1790714876974330 us; error 68 us; skew 500 ppm
I20260929 20:47:56.974898 1987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 20:47:56.977540 1987 webserver.cc:460] Webserver started at http://127.0.97.62:40179/ using document root <none> and password file <none>
I20260929 20:47:56.978302 1987 fs_manager.cc:359] Metadata directory not provided
I20260929 20:47:56.978448 1987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 20:47:56.983901 1987 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 20:47:56.987246 2001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 20:47:56.988826 1987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 20:47:56.989058 1987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data,/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8"
format_stamp: "Formatted at 2026-09-29 20:47:41 on dist-test-slave-j0bz"
I20260929 20:47:56.990079 1987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 20:47:57.009478 1987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 20:47:57.010432 1987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 20:47:57.010728 1987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 20:47:57.031751 1987 rpc_server.cc:307] RPC server started. Bound to: 127.0.97.62:46175
I20260929 20:47:57.031805 2052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.97.62:46175 every 8 connection(s)
I20260929 20:47:57.033360 1987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
I20260929 20:47:57.034194 388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu as pid 1987
I20260929 20:47:57.042037 2053 sys_catalog.cc:263] Verifying existing consensus state
I20260929 20:47:57.046440 2053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Bootstrap starting.
I20260929 20:47:57.058867 2053 log.cc:824] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Log is configured to *not* fsync() on all Append() calls
I20260929 20:47:57.093868 2053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=43 overwritten=0 applied=43 ignored=0} inserts{seen=29 ignored=0} mutations{seen=27 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 20:47:57.094429 2053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Bootstrap complete.
I20260929 20:47:57.101933 2053 raft_consensus.cc:348] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:57.103191 2053 raft_consensus.cc:729] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32aaf3821ff4ad09bcbfe11ff2e03c8, State: Initialized, Role: FOLLOWER
I20260929 20:47:57.103826 2053 consensus_queue.cc:260] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 43, Last appended: 6.43, Last appended by leader: 43, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:57.104676 2057 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:47:57.105014 2057 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:47:57.105849 2053 sys_catalog.cc:564] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 20:47:57.136010 2071 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:47:57.141414 2071 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: loading cluster ID for follower catalog manager: success
I20260929 20:47:57.147119 2071 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: acquiring CA information for follower catalog manager: success
I20260929 20:47:57.148000 1182 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:47:57.150250 2071 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 20:47:57.162199 2017 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" instance_seqno: 1790714863999745) as {username='slave'} at 127.0.97.2:57967; Asking this server to re-register.
I20260929 20:47:57.151827 1987 master_runner.cc:186] Error getting master registration for 127.0.97.61:34103: Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111),
I20260929 20:47:57.163205 1987 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:47:57.163805 1182 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:57.165684 2017 ts_manager.cc:194] Registered new tserver with Master: cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:47:57.348747 1324 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:47:57.349236 1037 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:47:57.350534 2017 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "1cb32e9690a546149d11339d371c832f" instance_seqno: 1790714864581790) as {username='slave'} at 127.0.97.3:51401; Asking this server to re-register.
I20260929 20:47:57.350632 2018 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" instance_seqno: 1790714863424856) as {username='slave'} at 127.0.97.1:56371; Asking this server to re-register.
I20260929 20:47:57.351320 1324 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:57.351507 1037 heartbeater.cc:464] Registering TS with master...
I20260929 20:47:57.352808 2018 ts_manager.cc:194] Registered new tserver with Master: 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:47:57.354082 2018 ts_manager.cc:194] Registered new tserver with Master: d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657)
I20260929 20:47:57.545806 2057 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:57.546278 2057 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:57.548343 2057 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:47:57.549234 2002 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
W20260929 20:47:57.551180 2004 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:47:57.551479 2002 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
I20260929 20:47:57.551692 2002 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8; no voters: bfbad3ef4593415191fa0d993866d0ea, cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:47:57.552132 2057 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260929 20:47:58.061187 2079 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:58.061424 2079 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:58.062104 2079 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:47:58.064436 2004 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:47:58.064677 2002 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
I20260929 20:47:58.064800 2002 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8; no voters: bfbad3ef4593415191fa0d993866d0ea, cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:47:58.065101 2079 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260929 20:47:59.104312 2083 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:59.104539 2083 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:47:59.105242 2083 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:47:59.107312 2002 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
W20260929 20:47:59.107568 2004 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:47:59.107726 2004 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8; no voters: bfbad3ef4593415191fa0d993866d0ea, cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:47:59.107995 2083 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260929 20:47:59.285626 2078 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:59.285969 2078 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:47:59.286666 2078 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:47:59.287660 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:47:59.287829 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:47:59.288015 925 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:47:59.288311 2078 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:47:59.479552 2078 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:59.479744 2078 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:47:59.480532 2078 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:47:59.481498 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:47:59.481696 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:47:59.481845 925 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:47:59.482089 2078 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:47:59.605091 388 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu pid 1987
I20260929 20:47:59.617883 2048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 20:47:59.728631 388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu with pid 1987
I20260929 20:47:59.728636 2091 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:47:59.728822 2091 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:00.745674 388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
/tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.62:46175
--webserver_interface=127.0.97.62
--webserver_port=40179
--builtin_ntp_servers=127.0.97.20:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260929 20:48:01.088416 2093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 20:48:01.088873 2093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 20:48:01.089035 2093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 20:48:01.096304 2093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 20:48:01.096447 2093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 20:48:01.096575 2093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 20:48:01.096669 2093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 20:48:01.106405 2093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.97.20:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.62:46175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.97.62
--webserver_port=40179
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 19:57:29 UTC on ccycloud.quasar-xxilbq.root.comops.site
build id 4681269
ASAN enabled
I20260929 20:48:01.107820 2093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 20:48:01.109704 2093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 20:48:01.122104 2099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:48:01.122149 2100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:48:01.122192 2102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 20:48:01.123270 2093 server_base.cc:1034] running on GCE node
I20260929 20:48:01.124102 2093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 20:48:01.125631 2093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 20:48:01.126853 2093 hybrid_clock.cc:648] HybridClock initialized: now 1790714881126783 us; error 45 us; skew 500 ppm
I20260929 20:48:01.127233 2093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 20:48:01.129599 2093 webserver.cc:460] Webserver started at http://127.0.97.62:40179/ using document root <none> and password file <none>
I20260929 20:48:01.130338 2093 fs_manager.cc:359] Metadata directory not provided
I20260929 20:48:01.130482 2093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 20:48:01.136456 2093 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260929 20:48:01.139617 2108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 20:48:01.141106 2093 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 20:48:01.141304 2093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data,/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8"
format_stamp: "Formatted at 2026-09-29 20:47:41 on dist-test-slave-j0bz"
I20260929 20:48:01.142292 2093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 20:48:01.155122 2093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 20:48:01.156041 2093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 20:48:01.156339 2093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 20:48:01.176324 2093 rpc_server.cc:307] RPC server started. Bound to: 127.0.97.62:46175
I20260929 20:48:01.176357 2159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.97.62:46175 every 8 connection(s)
I20260929 20:48:01.177874 2093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
I20260929 20:48:01.184485 2160 sys_catalog.cc:263] Verifying existing consensus state
I20260929 20:48:01.184875 388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu as pid 2093
I20260929 20:48:01.189320 2160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Bootstrap starting.
I20260929 20:48:01.203794 2160 log.cc:824] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Log is configured to *not* fsync() on all Append() calls
I20260929 20:48:01.237756 2160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=43 overwritten=0 applied=43 ignored=0} inserts{seen=29 ignored=0} mutations{seen=27 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 20:48:01.238353 2160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Bootstrap complete.
I20260929 20:48:01.247390 2160 raft_consensus.cc:348] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:01.248486 2160 raft_consensus.cc:729] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32aaf3821ff4ad09bcbfe11ff2e03c8, State: Initialized, Role: FOLLOWER
I20260929 20:48:01.249178 2160 consensus_queue.cc:260] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 43, Last appended: 6.43, Last appended by leader: 43, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:01.249994 2164 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:48:01.250305 2164 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:48:01.251039 2160 sys_catalog.cc:564] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 20:48:01.260371 1182 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:48:01.264230 2125 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" instance_seqno: 1790714863999745) as {username='slave'} at 127.0.97.2:38911; Asking this server to re-register.
I20260929 20:48:01.265259 1182 heartbeater.cc:464] Registering TS with master...
I20260929 20:48:01.267521 2125 ts_manager.cc:194] Registered new tserver with Master: cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:48:01.269356 2178 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:48:01.269541 2178 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: loading cluster ID for follower catalog manager: success
I20260929 20:48:01.275485 2178 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: acquiring CA information for follower catalog manager: success
I20260929 20:48:01.279876 2178 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 20:48:01.280220 2093 master_runner.cc:186] Error getting master registration for 127.0.97.61:34103: Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111),
I20260929 20:48:01.287889 2093 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:48:01.376139 1324 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:48:01.376546 1037 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:48:01.377807 2125 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "1cb32e9690a546149d11339d371c832f" instance_seqno: 1790714864581790) as {username='slave'} at 127.0.97.3:40403; Asking this server to re-register.
I20260929 20:48:01.378153 2124 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" instance_seqno: 1790714863424856) as {username='slave'} at 127.0.97.1:52827; Asking this server to re-register.
I20260929 20:48:01.378372 1324 heartbeater.cc:464] Registering TS with master...
I20260929 20:48:01.378829 1037 heartbeater.cc:464] Registering TS with master...
I20260929 20:48:01.379819 2124 ts_manager.cc:194] Registered new tserver with Master: 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:48:01.380352 2125 ts_manager.cc:194] Registered new tserver with Master: d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657)
I20260929 20:48:01.851095 2184 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:01.851351 2184 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:01.853292 2184 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:01.854370 2109 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
W20260929 20:48:01.855935 2111 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:48:01.856204 2109 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
I20260929 20:48:01.856391 2109 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8; no voters: bfbad3ef4593415191fa0d993866d0ea, cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:48:01.856887 2184 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260929 20:48:02.421168 2189 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:02.421382 2189 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:02.422119 2189 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:02.424562 2109 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
W20260929 20:48:02.424769 2111 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:02.424974 2111 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8; no voters: bfbad3ef4593415191fa0d993866d0ea, cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:48:02.425279 2189 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260929 20:48:03.248890 2193 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:03.249069 2193 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:03.249682 2193 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:03.251796 2111 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:48:03.252031 2109 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
I20260929 20:48:03.252156 2109 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8; no voters: bfbad3ef4593415191fa0d993866d0ea, cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:48:03.252382 2193 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260929 20:48:03.785913 2198 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:03.786201 2198 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:03.786900 2198 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:03.789359 2111 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:48:03.789552 2109 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
I20260929 20:48:03.789669 2109 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8; no voters: bfbad3ef4593415191fa0d993866d0ea, cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:48:03.789997 2198 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260929 20:48:04.286388 388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
/tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.61:34103
--webserver_interface=127.0.97.61
--webserver_port=36563
--builtin_ntp_servers=127.0.97.20:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260929 20:48:04.578761 2203 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:04.579010 2203 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:04.579784 2203 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:04.582319 2111 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:48:04.582538 2109 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103): Network error: Client connection negotiation failed: client connection to 127.0.97.61:34103: connect: Connection refused (error 111)
I20260929 20:48:04.582684 2109 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8; no voters: bfbad3ef4593415191fa0d993866d0ea, cc99fb1b059842088a50ba9c56fda6c1
I20260929 20:48:04.582983 2203 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260929 20:48:04.637938 2202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 20:48:04.638252 2202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 20:48:04.638396 2202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 20:48:04.646337 2202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 20:48:04.646452 2202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 20:48:04.646541 2202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 20:48:04.646617 2202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 20:48:04.656610 2202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.97.20:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.61:34103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.97.61
--webserver_port=36563
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 19:57:29 UTC on ccycloud.quasar-xxilbq.root.comops.site
build id 4681269
ASAN enabled
I20260929 20:48:04.658174 2202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 20:48:04.660111 2202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 20:48:04.672993 2212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:48:04.673015 2211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 20:48:04.673931 2202 server_base.cc:1034] running on GCE node
W20260929 20:48:04.673936 2214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 20:48:04.674962 2202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 20:48:04.676577 2202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 20:48:04.677845 2202 hybrid_clock.cc:648] HybridClock initialized: now 1790714884677787 us; error 46 us; skew 500 ppm
I20260929 20:48:04.678231 2202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 20:48:04.685087 2202 webserver.cc:460] Webserver started at http://127.0.97.61:36563/ using document root <none> and password file <none>
I20260929 20:48:04.685940 2202 fs_manager.cc:359] Metadata directory not provided
I20260929 20:48:04.686127 2202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 20:48:04.692374 2202 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.001s
I20260929 20:48:04.695871 2220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 20:48:04.697597 2202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 20:48:04.697826 2202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data,/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/wal
uuid: "cc99fb1b059842088a50ba9c56fda6c1"
format_stamp: "Formatted at 2026-09-29 20:47:41 on dist-test-slave-j0bz"
I20260929 20:48:04.698939 2202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 20:48:04.718422 2202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 20:48:04.719509 2202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 20:48:04.719837 2202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 20:48:04.740784 2202 rpc_server.cc:307] RPC server started. Bound to: 127.0.97.61:34103
I20260929 20:48:04.740801 2271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.97.61:34103 every 8 connection(s)
I20260929 20:48:04.742488 2202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data/info.pb
I20260929 20:48:04.746922 388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu as pid 2202
I20260929 20:48:04.750407 2272 sys_catalog.cc:263] Verifying existing consensus state
I20260929 20:48:04.754592 2272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: Bootstrap starting.
I20260929 20:48:04.767319 2272 log.cc:824] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: Log is configured to *not* fsync() on all Append() calls
I20260929 20:48:04.801736 2272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=39 overwritten=0 applied=39 ignored=0} inserts{seen=27 ignored=0} mutations{seen=25 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 20:48:04.802486 2272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: Bootstrap complete.
I20260929 20:48:04.814647 2272 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:04.816017 2272 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc99fb1b059842088a50ba9c56fda6c1, State: Initialized, Role: FOLLOWER
I20260929 20:48:04.816670 1038 heartbeater.cc:347] Connected to a master server at 127.0.97.61:34103
I20260929 20:48:04.816834 2272 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 39, Last appended: 4.39, Last appended by leader: 39, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:04.817773 2278 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:48:04.818192 2278 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:48:04.818701 1325 heartbeater.cc:347] Connected to a master server at 127.0.97.61:34103
I20260929 20:48:04.819141 2272 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 20:48:04.853456 2292 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:48:04.853606 2292 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: loading cluster ID for follower catalog manager: success
I20260929 20:48:04.857343 2292 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: acquiring CA information for follower catalog manager: success
I20260929 20:48:04.860424 2292 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 20:48:04.875000 2202 master_runner.cc:186] Error getting master registration for 127.0.97.60:45599: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111),
I20260929 20:48:04.884817 2202 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:48:05.117614 1183 heartbeater.cc:347] Connected to a master server at 127.0.97.61:34103
I20260929 20:48:05.122191 2236 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" instance_seqno: 1790714863999745) as {username='slave'} at 127.0.97.2:51349; Asking this server to re-register.
I20260929 20:48:05.124411 1183 heartbeater.cc:464] Registering TS with master...
I20260929 20:48:05.127292 2236 ts_manager.cc:194] Registered new tserver with Master: cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:48:05.323132 2236 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" instance_seqno: 1790714863424856) as {username='slave'} at 127.0.97.1:51129; Asking this server to re-register.
I20260929 20:48:05.323940 1038 heartbeater.cc:464] Registering TS with master...
I20260929 20:48:05.325487 2236 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "1cb32e9690a546149d11339d371c832f" instance_seqno: 1790714864581790) as {username='slave'} at 127.0.97.3:33633; Asking this server to re-register.
I20260929 20:48:05.325706 2237 ts_manager.cc:194] Registered new tserver with Master: d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657)
I20260929 20:48:05.326344 1325 heartbeater.cc:464] Registering TS with master...
I20260929 20:48:05.327841 2237 ts_manager.cc:194] Registered new tserver with Master: 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:48:05.338804 2299 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:05.339044 2299 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:05.340979 2299 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:05.341693 2223 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:05.341636 2135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc99fb1b059842088a50ba9c56fda6c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 39 } } ignore_live_leader: false dest_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" is_pre_election: true
I20260929 20:48:05.342113 2135 raft_consensus.cc:2354] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cc99fb1b059842088a50ba9c56fda6c1 for earlier term 5. Current term is 6.
I20260929 20:48:05.343046 2222 leader_election.cc:398] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 5 pre-election: Vote denied by peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cc99fb1b059842088a50ba9c56fda6c1 for earlier term 5. Current term is 6.
I20260929 20:48:05.343158 2222 leader_election.cc:401] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260929 20:48:05.343323 2223 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:05.343469 2299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 4 FOLLOWER]: Advancing to term 6
I20260929 20:48:05.347692 2299 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cc99fb1b059842088a50ba9c56fda6c1 for earlier term 5. Current term is 6.
I20260929 20:48:06.078790 2304 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:06.078986 2304 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:06.079713 2304 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
I20260929 20:48:06.080343 2135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc99fb1b059842088a50ba9c56fda6c1" candidate_term: 7 candidate_status { last_received { term: 4 index: 39 } } ignore_live_leader: false dest_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" is_pre_election: true
I20260929 20:48:06.080704 2135 raft_consensus.cc:2396] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cc99fb1b059842088a50ba9c56fda6c1 for term 7 because replica has last-logged OpId of term: 6 index: 43, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 39.
W20260929 20:48:06.082190 2223 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:06.082302 2223 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:06.082589 2304 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260929 20:48:06.300348 2093 master_runner.cc:186] Error getting master registration for 127.0.97.60:45599: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111),
I20260929 20:48:06.300639 2093 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:48:06.903724 2188 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:06.903929 2188 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:48:06.904997 2188 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:48:06.905877 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:48:06.906028 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:48:06.906126 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:48:06.906441 2188 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:48:06.910032 2315 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:06.910202 2315 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:06.910737 2315 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
I20260929 20:48:06.911319 2135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc99fb1b059842088a50ba9c56fda6c1" candidate_term: 7 candidate_status { last_received { term: 4 index: 39 } } ignore_live_leader: false dest_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" is_pre_election: true
I20260929 20:48:06.911645 2135 raft_consensus.cc:2396] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cc99fb1b059842088a50ba9c56fda6c1 for term 7 because replica has last-logged OpId of term: 6 index: 43, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 39.
W20260929 20:48:06.912926 2223 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:06.913079 2223 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:06.913333 2315 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260929 20:48:07.138445 388 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu pid 2093
I20260929 20:48:07.146328 2155 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 20:48:07.256935 2321 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:07.257001 388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu with pid 2093
I20260929 20:48:07.587644 2188 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:07.587924 2188 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:48:07.588656 2188 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:48:07.589735 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:48:07.589969 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:48:07.590103 924 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:48:07.590407 2188 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:48:07.689700 2325 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:07.689924 2325 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:07.690689 2325 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:07.693385 2222 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:48:07.693656 2223 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:07.693816 2223 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:07.694092 2325 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260929 20:48:07.868630 388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
/tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.62:46175
--webserver_interface=127.0.97.62
--webserver_port=40179
--builtin_ntp_servers=127.0.97.20:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260929 20:48:08.211798 2329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 20:48:08.212147 2329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 20:48:08.212404 2329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 20:48:08.220373 2329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 20:48:08.220573 2329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 20:48:08.220650 2329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 20:48:08.220767 2329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 20:48:08.231119 2329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.97.20:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.62:46175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.97.62
--webserver_port=40179
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 19:57:29 UTC on ccycloud.quasar-xxilbq.root.comops.site
build id 4681269
ASAN enabled
I20260929 20:48:08.232609 2329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 20:48:08.234570 2329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 20:48:08.246950 2334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:48:08.246838 2335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:48:08.246867 2337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 20:48:08.248262 2329 server_base.cc:1034] running on GCE node
I20260929 20:48:08.249248 2329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 20:48:08.250705 2329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 20:48:08.252008 2329 hybrid_clock.cc:648] HybridClock initialized: now 1790714888251873 us; error 44 us; skew 500 ppm
I20260929 20:48:08.252384 2329 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 20:48:08.254994 2329 webserver.cc:460] Webserver started at http://127.0.97.62:40179/ using document root <none> and password file <none>
I20260929 20:48:08.255792 2329 fs_manager.cc:359] Metadata directory not provided
I20260929 20:48:08.255939 2329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 20:48:08.261837 2329 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 20:48:08.265169 2343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 20:48:08.266719 2329 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260929 20:48:08.266927 2329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data,/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8"
format_stamp: "Formatted at 2026-09-29 20:47:41 on dist-test-slave-j0bz"
I20260929 20:48:08.267904 2329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 20:48:08.290874 2329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 20:48:08.291792 2329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 20:48:08.292126 2329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 20:48:08.310973 2329 rpc_server.cc:307] RPC server started. Bound to: 127.0.97.62:46175
I20260929 20:48:08.310979 2394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.97.62:46175 every 8 connection(s)
I20260929 20:48:08.312613 2329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
I20260929 20:48:08.318267 388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu as pid 2329
I20260929 20:48:08.320446 2395 sys_catalog.cc:263] Verifying existing consensus state
I20260929 20:48:08.326021 2395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Bootstrap starting.
I20260929 20:48:08.340205 2395 log.cc:824] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Log is configured to *not* fsync() on all Append() calls
I20260929 20:48:08.372473 2395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=43 overwritten=0 applied=43 ignored=0} inserts{seen=29 ignored=0} mutations{seen=27 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 20:48:08.373113 2395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Bootstrap complete.
I20260929 20:48:08.380595 2395 raft_consensus.cc:348] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:08.381494 2395 raft_consensus.cc:729] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32aaf3821ff4ad09bcbfe11ff2e03c8, State: Initialized, Role: FOLLOWER
I20260929 20:48:08.382093 2395 consensus_queue.cc:260] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 43, Last appended: 6.43, Last appended by leader: 43, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:08.383363 2399 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:48:08.383824 2399 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:48:08.384720 2395 sys_catalog.cc:564] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 20:48:08.399797 2413 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:48:08.399900 2413 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: loading cluster ID for follower catalog manager: success
I20260929 20:48:08.405176 2413 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: acquiring CA information for follower catalog manager: success
I20260929 20:48:08.408103 2413 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 20:48:08.419142 1037 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:48:08.423467 2360 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" instance_seqno: 1790714863424856) as {username='slave'} at 127.0.97.1:37847; Asking this server to re-register.
I20260929 20:48:08.425098 1037 heartbeater.cc:464] Registering TS with master...
I20260929 20:48:08.427525 2360 ts_manager.cc:194] Registered new tserver with Master: d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657)
I20260929 20:48:08.429301 1324 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:48:08.430778 2360 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "1cb32e9690a546149d11339d371c832f" instance_seqno: 1790714864581790) as {username='slave'} at 127.0.97.3:58847; Asking this server to re-register.
I20260929 20:48:08.431504 1324 heartbeater.cc:464] Registering TS with master...
I20260929 20:48:08.433015 2360 ts_manager.cc:194] Registered new tserver with Master: 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:48:08.436584 2329 master_runner.cc:186] Error getting master registration for 127.0.97.60:45599: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111),
I20260929 20:48:08.443605 2329 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:48:08.521335 2419 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:08.521591 2419 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:08.522351 2419 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:08.525689 2223 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:08.530224 2370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc99fb1b059842088a50ba9c56fda6c1" candidate_term: 7 candidate_status { last_received { term: 4 index: 39 } } ignore_live_leader: false dest_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" is_pre_election: true
I20260929 20:48:08.531134 2370 raft_consensus.cc:2396] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cc99fb1b059842088a50ba9c56fda6c1 for term 7 because replica has last-logged OpId of term: 6 index: 43, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 39.
I20260929 20:48:08.531760 2222 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:08.532092 2419 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260929 20:48:08.805832 1182 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:48:08.807101 2360 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" instance_seqno: 1790714863999745) as {username='slave'} at 127.0.97.2:52237; Asking this server to re-register.
I20260929 20:48:08.807694 1182 heartbeater.cc:464] Registering TS with master...
I20260929 20:48:08.808979 2360 ts_manager.cc:194] Registered new tserver with Master: cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:48:08.976847 2423 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:08.977296 2423 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:08.979410 2423 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
I20260929 20:48:08.980268 2247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 43 } } ignore_live_leader: false dest_uuid: "cc99fb1b059842088a50ba9c56fda6c1" is_pre_election: true
I20260929 20:48:08.980863 2247 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b32aaf3821ff4ad09bcbfe11ff2e03c8 in term 6.
W20260929 20:48:08.981408 2346 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:08.981498 2344 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, cc99fb1b059842088a50ba9c56fda6c1; no voters:
I20260929 20:48:08.981900 2423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260929 20:48:08.982168 2423 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:48:08.982306 2423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 6 FOLLOWER]: Advancing to term 7
W20260929 20:48:08.983106 2346 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:08.987022 2423 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:08.987824 2423 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 election: Requested vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
I20260929 20:48:08.988487 2247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 43 } } ignore_live_leader: false dest_uuid: "cc99fb1b059842088a50ba9c56fda6c1"
I20260929 20:48:08.988782 2247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 6 FOLLOWER]: Advancing to term 7
W20260929 20:48:08.989437 2346 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:08.990969 2247 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b32aaf3821ff4ad09bcbfe11ff2e03c8 in term 7.
I20260929 20:48:08.991514 2344 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, cc99fb1b059842088a50ba9c56fda6c1; no voters: bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:08.991896 2423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 7 FOLLOWER]: Leader election won for term 7
I20260929 20:48:08.992422 2423 raft_consensus.cc:686] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 7 LEADER]: Becoming Leader. State: Replica: b32aaf3821ff4ad09bcbfe11ff2e03c8, State: Running, Role: LEADER
I20260929 20:48:08.992980 2423 consensus_queue.cc:237] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 6.43, Last appended by leader: 43, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:08.995740 2426 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b32aaf3821ff4ad09bcbfe11ff2e03c8. Latest consensus state: current_term: 7 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:48:08.996168 2426 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: LEADER
I20260929 20:48:08.996974 2428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 20:48:08.999709 2428 catalog_manager.cc:669] Loaded metadata for table default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]
I20260929 20:48:09.000152 2428 catalog_manager.cc:669] Loaded metadata for table default.table_5b5c251fd82449869826c01b42a62f31 [id=3842db9126434978a3631262acd85d32]
I20260929 20:48:09.000428 2428 catalog_manager.cc:669] Loaded metadata for table default.table_0b803212f31d45419130e1da10d1a620 [id=796a2c5810ac4950a0f9499208fad0e9]
I20260929 20:48:09.000922 2428 catalog_manager.cc:669] Loaded metadata for table default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f]
I20260929 20:48:09.001235 2428 catalog_manager.cc:669] Loaded metadata for table default.table_556595b54fef48f99c72c48e730495e9 [id=b17facaee1924496b1beaf3d695c11ab]
I20260929 20:48:09.001530 2428 catalog_manager.cc:669] Loaded metadata for table default.table_f02a75c1e46d472d90334b4340d99e7b [id=dd1c5cbd5c6c40fc9fb5cc1dbe7026b0]
I20260929 20:48:09.001776 2428 catalog_manager.cc:669] Loaded metadata for table default.table_a5aa493381b543419904f1429ffc19f2 [id=e7f4bcce9df4454a81f6e98a5b2e69e5]
I20260929 20:48:09.002044 2428 catalog_manager.cc:669] Loaded metadata for table default.table_952135bfbb564f519938b6b206b1d463 [id=f04d505b1ef9453780f36b454bc22f46]
I20260929 20:48:09.002318 2428 catalog_manager.cc:669] Loaded metadata for table default.table_47f3dc18fe564237bda167352f1afa08 [id=f825b412c46842ec967df7bb7d8f9474]
I20260929 20:48:09.004132 2428 tablet_loader.cc:96] loaded metadata for tablet 061c178e50924b51a92542bea703b24c (table default.table_556595b54fef48f99c72c48e730495e9 [id=b17facaee1924496b1beaf3d695c11ab])
I20260929 20:48:09.004875 2428 tablet_loader.cc:96] loaded metadata for tablet 42279556b25240deaf8a689465368262 (table default.table_0b803212f31d45419130e1da10d1a620 [id=796a2c5810ac4950a0f9499208fad0e9])
I20260929 20:48:09.005429 2428 tablet_loader.cc:96] loaded metadata for tablet 5e20f360f9b74f3f929b7acfdf82671f (table default.table_47f3dc18fe564237bda167352f1afa08 [id=f825b412c46842ec967df7bb7d8f9474])
I20260929 20:48:09.005692 2428 tablet_loader.cc:96] loaded metadata for tablet 6820644bb2a745679d959d8ac6854014 (table default.table_f02a75c1e46d472d90334b4340d99e7b [id=dd1c5cbd5c6c40fc9fb5cc1dbe7026b0])
I20260929 20:48:09.006186 2428 tablet_loader.cc:96] loaded metadata for tablet 9780cc79845d46d780893950ca742c02 (table default.table_a5aa493381b543419904f1429ffc19f2 [id=e7f4bcce9df4454a81f6e98a5b2e69e5])
I20260929 20:48:09.006606 2428 tablet_loader.cc:96] loaded metadata for tablet b6df369f96d54b33997ee5d4b4860df9 (table default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a])
I20260929 20:48:09.006848 2428 tablet_loader.cc:96] loaded metadata for tablet c2861c34ac464b9993b200df3fc201a1 (table default.table_5b5c251fd82449869826c01b42a62f31 [id=3842db9126434978a3631262acd85d32])
I20260929 20:48:09.007077 2428 tablet_loader.cc:96] loaded metadata for tablet c4a57da4e5c24803b7aa3f61c7191e38 (table default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f])
I20260929 20:48:09.007265 2428 tablet_loader.cc:96] loaded metadata for tablet c4c5549f5e28423386e92219593420ec (table default.table_952135bfbb564f519938b6b206b1d463 [id=f04d505b1ef9453780f36b454bc22f46])
I20260929 20:48:09.007625 2428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 20:48:09.008699 2428 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:48:09.008769 2428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 20:48:09.010067 2428 catalog_manager.cc:1504] Loading token signing keys...
I20260929 20:48:09.010957 2428 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Loaded TSK: 0
I20260929 20:48:09.011848 2428 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 20:48:09.013005 2428 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260929 20:48:09.014046 2428 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260929 20:48:09.049417 2360 master_service.cc:946] ReplaceTablet: received request to replace tablet b6df369f96d54b33997ee5d4b4860df9 from {username='slave'} at 127.0.0.1:43756
I20260929 20:48:09.049494 2359 master_service.cc:946] ReplaceTablet: received request to replace tablet 9780cc79845d46d780893950ca742c02 from {username='slave'} at 127.0.0.1:43756
I20260929 20:48:09.049965 2356 master_service.cc:946] ReplaceTablet: received request to replace tablet c4a57da4e5c24803b7aa3f61c7191e38 from {username='slave'} at 127.0.0.1:43756
I20260929 20:48:09.050029 2358 master_service.cc:946] ReplaceTablet: received request to replace tablet b6df369f96d54b33997ee5d4b4860df9 from {username='slave'} at 127.0.0.1:43756
I20260929 20:48:09.050364 2357 master_service.cc:946] ReplaceTablet: received request to replace tablet 6820644bb2a745679d959d8ac6854014 from {username='slave'} at 127.0.0.1:43756
I20260929 20:48:09.052542 2355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43756:
table { table_name: "default.Table_47f3dc18fe564237bda167352f1afa08" } modify_external_catalogs: true
I20260929 20:48:09.052914 2355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43756:
table { table_name: "default.Table_47f3dc18fe564237bda167352f1afa08" } modify_external_catalogs: true
I20260929 20:48:09.053857 2354 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43756:
name: "default.Table_031408b68a614af6b6ef8a7fe493dd75"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:48:09.054828 2353 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43756:
name: "default.Table_0d85111ed1f34848b4c76ba1a254e00f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:48:09.055743 2351 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43756:
table { table_name: "default.Table_952135bfbb564f519938b6b206b1d463" } new_table_name: "default.Table_c7abc2a81b1e45a682a347c05afa8fc7" modify_external_catalogs: true
I20260929 20:48:09.055948 2352 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43756:
name: "default.Table_21a7c2dce9cf486580fad3131b93aa63"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:48:09.059242 2247 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Refusing update from remote peer b32aaf3821ff4ad09bcbfe11ff2e03c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 39. Preceding OpId from leader: term: 7 index: 46. (index mismatch)
I20260929 20:48:09.060560 2426 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 39, Time since last communication: 0.000s
W20260929 20:48:09.061185 2346 consensus_peers.cc:487] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 -> Peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Couldn't send request to peer bfbad3ef4593415191fa0d993866d0ea. Status: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
F20260929 20:48:09.064175 2412 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_4cf3799777554d47b5dbb898a6d368b9
*** Check failure stack trace: ***
@ 0x7f8a03e719ad google::LogMessage::Fail() at ??:0
@ 0x7f8a03e7488a google::LogMessage::SendToLog() at ??:0
@ 0x7f8a03e714de google::LogMessage::Flush() at ??:0
@ 0x7f8a03e716d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f8a19a0fba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f8a19a0e901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f8a19a1566c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f8a19a154e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260929 20:48:09.100929 2436 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:48:09.101331 2436 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:48:09.101291 2440 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:48:09.101431 2429 mvcc.cc:205] Tried to move back new op lower bound from 7334768185573265408 to 7334768185318912000. Current Snapshot: MvccSnapshot[applied={T|T < 7334768185573265408}]
I20260929 20:48:09.101596 2440 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:48:09.100936 2431 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b32aaf3821ff4ad09bcbfe11ff2e03c8. Latest consensus state: current_term: 7 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:48:09.101779 2431 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f8a06a9beaf kudu::Thread::SuperviseThread() at ??:0
I20260929 20:48:09.107746 2431 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:48:09.108031 2431 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:48:09.108412 2356 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Sending DeleteTablet for 3 replicas of tablet c4a57da4e5c24803b7aa3f61c7191e38
@ 0x7f8a09d756db start_thread at ??:0
@ 0x7f8a02d0b61f clone at ??:0
W20260929 20:48:09.290731 2446 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 20:48:09.110810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48294 (local address 127.0.97.1:45657)
0929 20:48:09.114022 (+ 3212us) server_negotiation.cc:206] Beginning negotiation
0929 20:48:09.114026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 20:48:09.114048 (+ 22us) server_negotiation.cc:407] Connection header received
0929 20:48:09.114110 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 20:48:09.114113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 20:48:09.114173 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 20:48:09.114259 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 20:48:09.290316 (+176057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48294: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":3097,"thread_start_us":2970,"threads_started":1}
W20260929 20:48:09.290767 2447 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 20:48:09.110922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41360 (local address 127.0.97.2:41081)
0929 20:48:09.113921 (+ 2999us) server_negotiation.cc:206] Beginning negotiation
0929 20:48:09.113928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 20:48:09.290313 (+176385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41360: BlockingRecv error: recv got EOF from 127.0.0.1:41360 (error 108)
Metrics: {"server-negotiator.queue_time_us":2865,"thread_start_us":2684,"threads_started":1}
W20260929 20:48:09.291694 1332 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.97.62:46175)
W20260929 20:48:09.291985 1332 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.97.62:46175)
W20260929 20:48:09.292994 1332 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.97.62:46175)
W20260929 20:48:09.293233 1332 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.97.62:46175)
I20260929 20:48:09.610273 2457 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:48:09.610455 2457 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:09.611229 2457 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:09.613579 2222 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:48:09.613915 2223 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:09.614090 2223 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:09.614392 2457 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260929 20:48:09.886238 2202 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:48:09.886479 2202 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:48:10.290382 2461 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:48:10.290570 2461 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:10.291302 2461 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:10.293444 2222 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:48:10.293678 2223 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:10.293836 2223 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:10.294107 2461 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260929 20:48:10.738636 388 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu pid 2202
I20260929 20:48:10.746852 2267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 20:48:10.863013 388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu with pid 2202
I20260929 20:48:11.638973 388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
/tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.61:34103
--webserver_interface=127.0.97.61
--webserver_port=36563
--builtin_ntp_servers=127.0.97.20:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260929 20:48:11.973511 2470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 20:48:11.973887 2470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 20:48:11.974025 2470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 20:48:11.981974 2470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 20:48:11.982133 2470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 20:48:11.982228 2470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 20:48:11.982329 2470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 20:48:11.991958 2470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.97.20:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.61:34103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.97.61
--webserver_port=36563
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 19:57:29 UTC on ccycloud.quasar-xxilbq.root.comops.site
build id 4681269
ASAN enabled
I20260929 20:48:11.993386 2470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 20:48:11.995244 2470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 20:48:12.010229 2476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:48:12.010164 2477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:48:12.010185 2479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 20:48:12.011642 2470 server_base.cc:1034] running on GCE node
I20260929 20:48:12.012769 2470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 20:48:12.014940 2470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 20:48:12.016247 2470 hybrid_clock.cc:648] HybridClock initialized: now 1790714892016148 us; error 69 us; skew 500 ppm
I20260929 20:48:12.016736 2470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 20:48:12.020015 2470 webserver.cc:460] Webserver started at http://127.0.97.61:36563/ using document root <none> and password file <none>
I20260929 20:48:12.021036 2470 fs_manager.cc:359] Metadata directory not provided
I20260929 20:48:12.021180 2470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 20:48:12.029030 2470 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 20:48:12.032409 2485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 20:48:12.034168 2470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 20:48:12.034394 2470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data,/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/wal
uuid: "cc99fb1b059842088a50ba9c56fda6c1"
format_stamp: "Formatted at 2026-09-29 20:47:41 on dist-test-slave-j0bz"
I20260929 20:48:12.035405 2470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 20:48:12.059324 2470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 20:48:12.060271 2470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 20:48:12.060600 2470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 20:48:12.082849 2470 rpc_server.cc:307] RPC server started. Bound to: 127.0.97.61:34103
I20260929 20:48:12.082854 2536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.97.61:34103 every 8 connection(s)
I20260929 20:48:12.084467 2470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-1/data/info.pb
I20260929 20:48:12.088073 388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu as pid 2470
I20260929 20:48:12.092788 2537 sys_catalog.cc:263] Verifying existing consensus state
I20260929 20:48:12.096992 2537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: Bootstrap starting.
I20260929 20:48:12.109954 2537 log.cc:824] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: Log is configured to *not* fsync() on all Append() calls
I20260929 20:48:12.146128 2537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=48 overwritten=0 applied=48 ignored=0} inserts{seen=33 ignored=0} mutations{seen=31 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 20:48:12.146636 2537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: Bootstrap complete.
I20260929 20:48:12.154150 2537 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:12.155587 2537 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc99fb1b059842088a50ba9c56fda6c1, State: Initialized, Role: FOLLOWER
I20260929 20:48:12.156306 2537 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 48, Last appended: 7.48, Last appended by leader: 48, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:12.157231 2541 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:48:12.157663 2541 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:48:12.158557 2537 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 20:48:12.170583 1183 heartbeater.cc:347] Connected to a master server at 127.0.97.61:34103
I20260929 20:48:12.175149 2502 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" instance_seqno: 1790714863999745) as {username='slave'} at 127.0.97.2:50819; Asking this server to re-register.
I20260929 20:48:12.175382 2556 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:48:12.177302 2556 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: loading cluster ID for follower catalog manager: success
I20260929 20:48:12.178373 1183 heartbeater.cc:464] Registering TS with master...
I20260929 20:48:12.181115 2501 ts_manager.cc:194] Registered new tserver with Master: cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:48:12.182559 2556 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: acquiring CA information for follower catalog manager: success
I20260929 20:48:12.185230 2556 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 20:48:12.180274 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:48:12.190209 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:48:12.396195 1038 heartbeater.cc:347] Connected to a master server at 127.0.97.61:34103
I20260929 20:48:12.396324 1325 heartbeater.cc:347] Connected to a master server at 127.0.97.61:34103
I20260929 20:48:12.397917 2502 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" instance_seqno: 1790714863424856) as {username='slave'} at 127.0.97.1:53347; Asking this server to re-register.
I20260929 20:48:12.397917 2501 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "1cb32e9690a546149d11339d371c832f" instance_seqno: 1790714864581790) as {username='slave'} at 127.0.97.3:45571; Asking this server to re-register.
I20260929 20:48:12.398685 1325 heartbeater.cc:464] Registering TS with master...
I20260929 20:48:12.398702 1038 heartbeater.cc:464] Registering TS with master...
I20260929 20:48:12.400091 2502 ts_manager.cc:194] Registered new tserver with Master: 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:48:12.400301 2501 ts_manager.cc:194] Registered new tserver with Master: d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657)
I20260929 20:48:12.686703 2561 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:12.687004 2561 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:12.688850 2561 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:12.689823 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:48:12.691214 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:48:12.691445 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:12.691635 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:12.692025 2561 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260929 20:48:12.875080 2565 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:12.875285 2565 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:48:12.875993 2565 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:48:12.877012 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:48:12.877219 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:48:12.877316 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:48:12.877521 2565 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:48:13.263804 2566 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:13.263980 2566 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:13.264804 2566 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:13.267333 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:48:13.267581 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:48:13.267735 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:13.267997 2566 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260929 20:48:14.004252 2571 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:14.004586 2571 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:14.005297 2571 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:14.007609 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:48:14.007793 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:14.007918 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:14.008185 2571 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260929 20:48:15.231885 2575 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:15.232089 2575 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:15.232797 2575 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:15.235054 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:48:15.235250 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:15.235424 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:15.235709 2575 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260929 20:48:15.355197 388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
/tmp/dist-test-taskXtbUDl/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.62:46175
--webserver_interface=127.0.97.62
--webserver_port=40179
--builtin_ntp_servers=127.0.97.20:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260929 20:48:15.710273 2579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 20:48:15.710588 2579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 20:48:15.710664 2579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 20:48:15.718382 2579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 20:48:15.718505 2579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 20:48:15.718546 2579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 20:48:15.718604 2579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 20:48:15.729286 2579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.97.20:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:37237
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.97.62:46175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.97.62
--webserver_port=40179
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 19:57:29 UTC on ccycloud.quasar-xxilbq.root.comops.site
build id 4681269
ASAN enabled
I20260929 20:48:15.730621 2579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 20:48:15.732313 2579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 20:48:15.745666 2588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:48:15.745736 2586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 20:48:15.746134 2585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 20:48:15.746857 2579 server_base.cc:1034] running on GCE node
I20260929 20:48:15.747792 2579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 20:48:15.749584 2579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 20:48:15.750845 2579 hybrid_clock.cc:648] HybridClock initialized: now 1790714895750776 us; error 45 us; skew 500 ppm
I20260929 20:48:15.751219 2579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 20:48:15.758982 2579 webserver.cc:460] Webserver started at http://127.0.97.62:40179/ using document root <none> and password file <none>
I20260929 20:48:15.759824 2579 fs_manager.cc:359] Metadata directory not provided
I20260929 20:48:15.759972 2579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 20:48:15.765676 2579 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260929 20:48:15.768906 2594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 20:48:15.770412 2579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 20:48:15.770624 2579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data,/tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8"
format_stamp: "Formatted at 2026-09-29 20:47:41 on dist-test-slave-j0bz"
I20260929 20:48:15.771657 2579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 20:48:15.793437 2579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 20:48:15.794361 2579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 20:48:15.794703 2579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 20:48:15.814390 2645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.97.62:46175 every 8 connection(s)
I20260929 20:48:15.814385 2579 rpc_server.cc:307] RPC server started. Bound to: 127.0.97.62:46175
I20260929 20:48:15.815944 2579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXtbUDl/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790714844312622-388-0/minicluster-data/master-0/data/info.pb
I20260929 20:48:15.816610 388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXtbUDl/build/asan/bin/kudu as pid 2579
I20260929 20:48:15.824824 2646 sys_catalog.cc:263] Verifying existing consensus state
I20260929 20:48:15.830135 2646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Bootstrap starting.
I20260929 20:48:15.837929 1182 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:48:15.847259 2646 log.cc:824] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Log is configured to *not* fsync() on all Append() calls
I20260929 20:48:15.882622 2646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=48 overwritten=0 applied=45 ignored=0} inserts{seen=30 ignored=0} mutations{seen=28 ignored=0} orphaned_commits=0. Pending: 3 replicates
I20260929 20:48:15.883234 2646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Bootstrap complete.
I20260929 20:48:15.891180 2646 raft_consensus.cc:348] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 7 FOLLOWER]: Replica starting. Triggering 3 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:15.894717 2646 raft_consensus.cc:729] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32aaf3821ff4ad09bcbfe11ff2e03c8, State: Initialized, Role: FOLLOWER
I20260929 20:48:15.895437 2646 consensus_queue.cc:260] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 45, Last appended: 7.48, Last appended by leader: 48, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:15.896230 2653 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:48:15.896596 2653 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:48:15.897460 2646 sys_catalog.cc:564] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 20:48:15.912637 2667 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:48:15.912842 2667 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: loading cluster ID for follower catalog manager: success
I20260929 20:48:15.922087 2667 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: acquiring CA information for follower catalog manager: success
I20260929 20:48:15.925190 2667 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 20:48:15.944288 2579 master_runner.cc:186] Error getting master registration for 127.0.97.60:45599: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111),
I20260929 20:48:15.952801 2579 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:48:15.958494 1037 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:48:15.958825 1324 heartbeater.cc:347] Connected to a master server at 127.0.97.62:46175
I20260929 20:48:15.961874 2611 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" instance_seqno: 1790714863424856) as {username='slave'} at 127.0.97.1:45001; Asking this server to re-register.
I20260929 20:48:15.961979 2610 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "1cb32e9690a546149d11339d371c832f" instance_seqno: 1790714864581790) as {username='slave'} at 127.0.97.3:53161; Asking this server to re-register.
I20260929 20:48:15.962831 1324 heartbeater.cc:464] Registering TS with master...
I20260929 20:48:15.965788 2610 ts_manager.cc:194] Registered new tserver with Master: 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
I20260929 20:48:15.967219 1037 heartbeater.cc:464] Registering TS with master...
I20260929 20:48:15.968853 2610 ts_manager.cc:194] Registered new tserver with Master: d1f79a0bcbdb42fe89e7812efeae1a86 (127.0.97.1:45657)
I20260929 20:48:16.344434 2610 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" instance_seqno: 1790714863999745) as {username='slave'} at 127.0.97.2:55167; Asking this server to re-register.
I20260929 20:48:16.345418 1182 heartbeater.cc:464] Registering TS with master...
I20260929 20:48:16.346988 2610 ts_manager.cc:194] Registered new tserver with Master: cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
I20260929 20:48:16.367667 2653 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:16.367973 2653 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:16.369997 2653 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:16.370812 2597 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:16.370720 2512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 48 } } ignore_live_leader: false dest_uuid: "cc99fb1b059842088a50ba9c56fda6c1" is_pre_election: true
I20260929 20:48:16.371179 2512 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b32aaf3821ff4ad09bcbfe11ff2e03c8 in term 7.
I20260929 20:48:16.371789 2595 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, cc99fb1b059842088a50ba9c56fda6c1; no voters:
I20260929 20:48:16.372140 2653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260929 20:48:16.372277 2653 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 20:48:16.372397 2653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 7 FOLLOWER]: Advancing to term 8
W20260929 20:48:16.372486 2597 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:16.376157 2653 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:16.376943 2653 leader_election.cc:288] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 8 election: Requested vote from peers cc99fb1b059842088a50ba9c56fda6c1 (127.0.97.61:34103), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
I20260929 20:48:16.377455 2512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 48 } } ignore_live_leader: false dest_uuid: "cc99fb1b059842088a50ba9c56fda6c1"
I20260929 20:48:16.377691 2512 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 7 FOLLOWER]: Advancing to term 8
W20260929 20:48:16.378417 2597 leader_election.cc:334] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:16.381691 2512 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b32aaf3821ff4ad09bcbfe11ff2e03c8 in term 8.
I20260929 20:48:16.382205 2595 leader_election.cc:302] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, cc99fb1b059842088a50ba9c56fda6c1; no voters: bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:16.382534 2653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 8 FOLLOWER]: Leader election won for term 8
I20260929 20:48:16.382951 2653 raft_consensus.cc:686] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [term 8 LEADER]: Becoming Leader. State: Replica: b32aaf3821ff4ad09bcbfe11ff2e03c8, State: Running, Role: LEADER
I20260929 20:48:16.383404 2653 consensus_queue.cc:237] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 45, Committed index: 45, Last appended: 7.48, Last appended by leader: 48, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:16.386231 2673 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b32aaf3821ff4ad09bcbfe11ff2e03c8. Latest consensus state: current_term: 8 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:48:16.386700 2673 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: LEADER
W20260929 20:48:16.818779 2597 consensus_peers.cc:487] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 -> Peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Couldn't send request to peer bfbad3ef4593415191fa0d993866d0ea. Status: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 20:48:16.966737 2512 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Refusing update from remote peer b32aaf3821ff4ad09bcbfe11ff2e03c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 48. Preceding OpId from leader: term: 8 index: 49. (index mismatch)
I20260929 20:48:16.967664 2673 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 48, Time since last communication: 0.000s
I20260929 20:48:16.971506 2680 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b32aaf3821ff4ad09bcbfe11ff2e03c8. Latest consensus state: current_term: 8 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:48:16.971873 2680 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:48:16.977596 2680 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 8 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:48:16.977895 2680 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 20:48:16.978955 2688 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 8 leader_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } } }
I20260929 20:48:16.979162 2688 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8 [sys.catalog]: This master's current role is: LEADER
I20260929 20:48:17.032418 2675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 20:48:17.035065 2675 catalog_manager.cc:669] Loaded metadata for table default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a]
I20260929 20:48:17.035521 2675 catalog_manager.cc:669] Loaded metadata for table default.table_5b5c251fd82449869826c01b42a62f31 [id=3842db9126434978a3631262acd85d32]
I20260929 20:48:17.035797 2675 catalog_manager.cc:669] Loaded metadata for table default.table_0b803212f31d45419130e1da10d1a620 [id=796a2c5810ac4950a0f9499208fad0e9]
I20260929 20:48:17.036062 2675 catalog_manager.cc:669] Loaded metadata for table default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f]
I20260929 20:48:17.036350 2675 catalog_manager.cc:669] Loaded metadata for table default.table_556595b54fef48f99c72c48e730495e9 [id=b17facaee1924496b1beaf3d695c11ab]
I20260929 20:48:17.036738 2675 catalog_manager.cc:669] Loaded metadata for table default.table_f02a75c1e46d472d90334b4340d99e7b [id=dd1c5cbd5c6c40fc9fb5cc1dbe7026b0]
I20260929 20:48:17.037065 2675 catalog_manager.cc:669] Loaded metadata for table default.table_a5aa493381b543419904f1429ffc19f2 [id=e7f4bcce9df4454a81f6e98a5b2e69e5]
I20260929 20:48:17.037520 2675 catalog_manager.cc:669] Loaded metadata for table default.table_952135bfbb564f519938b6b206b1d463 [id=f04d505b1ef9453780f36b454bc22f46]
I20260929 20:48:17.038347 2675 catalog_manager.cc:669] Loaded metadata for table default.table_47f3dc18fe564237bda167352f1afa08 [id=f825b412c46842ec967df7bb7d8f9474]
I20260929 20:48:17.040311 2675 tablet_loader.cc:96] loaded metadata for tablet 061c178e50924b51a92542bea703b24c (table default.table_556595b54fef48f99c72c48e730495e9 [id=b17facaee1924496b1beaf3d695c11ab])
I20260929 20:48:17.040887 2675 tablet_loader.cc:96] loaded metadata for tablet 22f3984666f54d448b2b186027943f29 (table default.table_ef3bd0e733d74bf8a3cc021239e8dea5 [id=843416a726514cbfb345820f1877b73f])
I20260929 20:48:17.041301 2675 tablet_loader.cc:96] loaded metadata for tablet 42279556b25240deaf8a689465368262 (table default.table_0b803212f31d45419130e1da10d1a620 [id=796a2c5810ac4950a0f9499208fad0e9])
I20260929 20:48:17.041674 2675 tablet_loader.cc:96] loaded metadata for tablet 530afde0d00441a2a5eed612238c7961 (table default.table_a5aa493381b543419904f1429ffc19f2 [id=e7f4bcce9df4454a81f6e98a5b2e69e5])
I20260929 20:48:17.041945 2675 tablet_loader.cc:96] loaded metadata for tablet 53357ec7bef74a5a88ed8435fea5dcf1 (table default.table_f02a75c1e46d472d90334b4340d99e7b [id=dd1c5cbd5c6c40fc9fb5cc1dbe7026b0])
I20260929 20:48:17.042371 2675 tablet_loader.cc:96] loaded metadata for tablet 5e20f360f9b74f3f929b7acfdf82671f (table default.table_47f3dc18fe564237bda167352f1afa08 [id=f825b412c46842ec967df7bb7d8f9474])
I20260929 20:48:17.042783 2675 tablet_loader.cc:96] loaded metadata for tablet 771198720f1f4b99b4b311e0503fb09c (table default.table_d9ce7635a33b4f6ea325f294cc7db677 [id=274aedc84b104ba38dee5cfff431905a])
I20260929 20:48:17.043694 2675 tablet_loader.cc:96] loaded metadata for tablet c2861c34ac464b9993b200df3fc201a1 (table default.table_5b5c251fd82449869826c01b42a62f31 [id=3842db9126434978a3631262acd85d32])
I20260929 20:48:17.044086 2675 tablet_loader.cc:96] loaded metadata for tablet c4c5549f5e28423386e92219593420ec (table default.table_952135bfbb564f519938b6b206b1d463 [id=f04d505b1ef9453780f36b454bc22f46])
I20260929 20:48:17.044536 2675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 20:48:17.045763 2675 catalog_manager.cc:1259] Loaded cluster ID: aab167fa70c44b1d94dd39a05d8af783
I20260929 20:48:17.045861 2675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 20:48:17.047534 2675 catalog_manager.cc:1504] Loading token signing keys...
I20260929 20:48:17.048628 2675 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b32aaf3821ff4ad09bcbfe11ff2e03c8: Loaded TSK: 0
I20260929 20:48:17.049624 2675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 20:48:17.050523 2675 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260929 20:48:17.051373 2675 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260929 20:48:17.175576 2611 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50044:
name: "default.Table_a988b1827f9e482086b9b9ad0663c2b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:48:17.175194 2608 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50044:
name: "default.Table_be3c404093574a04aea5256b4ecba3bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:48:17.175652 2605 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50044:
name: "default.Table_dae8faea7e2b44909906d91d85898732"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:48:17.175385 2607 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50044:
name: "default.Table_57d2f2bbf5c3465ba1591d6a2b6fbbc0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:48:17.174844 2610 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50044:
table { table_name: "default.Table_0b803212f31d45419130e1da10d1a620" } new_table_name: "default.Table_71dc24415d9046608c47e70ce90636c4" modify_external_catalogs: true
I20260929 20:48:17.175500 2606 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50044:
name: "default.Table_e3c4d53b991348e8af5e335ad5847221"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:48:17.175459 2609 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50044:
name: "default.Table_e1a3cd10e94d42c598e9c071db2144d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:48:17.177961 2602 master_service.cc:946] ReplaceTablet: received request to replace tablet c4a57da4e5c24803b7aa3f61c7191e38 from {username='slave'} at 127.0.0.1:50044
I20260929 20:48:17.177402 2603 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50044:
table { table_name: "default.Table_47f3dc18fe564237bda167352f1afa08" } modify_external_catalogs: true
I20260929 20:48:17.178897 2603 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50044:
table { table_name: "default.Table_47f3dc18fe564237bda167352f1afa08" } modify_external_catalogs: true
I20260929 20:48:17.178802 2604 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50044:
name: "default.Table_55d763f6ec744322a43d80efeb51983e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 20:48:17.179203 2602 master_service.cc:946] ReplaceTablet: received request to replace tablet 6820644bb2a745679d959d8ac6854014 from {username='slave'} at 127.0.0.1:50044
I20260929 20:48:17.179699 2602 master_service.cc:946] ReplaceTablet: received request to replace tablet b6df369f96d54b33997ee5d4b4860df9 from {username='slave'} at 127.0.0.1:50044
I20260929 20:48:17.180096 2602 master_service.cc:946] ReplaceTablet: received request to replace tablet b6df369f96d54b33997ee5d4b4860df9 from {username='slave'} at 127.0.0.1:50044
I20260929 20:48:17.180550 2602 master_service.cc:946] ReplaceTablet: received request to replace tablet 9780cc79845d46d780893950ca742c02 from {username='slave'} at 127.0.0.1:50044
I20260929 20:48:17.181049 2602 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50044:
name: "default.Table_031408b68a614af6b6ef8a7fe493dd75"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260929 20:48:17.183435 2666 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_4cf3799777554d47b5dbb898a6d368b9
*** Check failure stack trace: ***
@ 0x7fdb8def89ad google::LogMessage::Fail() at ??:0
@ 0x7fdb8defb88a google::LogMessage::SendToLog() at ??:0
@ 0x7fdb8def84de google::LogMessage::Flush() at ??:0
@ 0x7fdb8def86d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7fdba3a96ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7fdba3a95901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7fdba3a9c66c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7fdba3a9c4e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fdb90b22eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fdb93dfc6db start_thread at ??:0
@ 0x7fdb8cd9261f clone at ??:0
I20260929 20:48:17.396474 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: recv got EOF from 127.0.97.62:46175 (error 108),
I20260929 20:48:17.396737 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 20:48:17.396910 1070 connection.cc:535] client connection to 127.0.97.62:46175 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 20:48:17.397410 1332 connection.cc:535] client connection to 127.0.97.62:46175 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 20:48:17.398183 1332 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.97.62:46175)
W20260929 20:48:17.398926 1332 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.97.62:46175)
W20260929 20:48:17.399331 1332 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.97.62:46175)
W20260929 20:48:17.399748 1332 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.97.62:46175)
I20260929 20:48:17.429870 2676 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:17.430083 2676 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:48:17.430685 2676 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:48:17.431645 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:48:17.431888 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:48:17.432013 925 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:48:17.432296 2676 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:48:17.475997 2680 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:48:17.476215 2680 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:17.476984 2680 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:17.479141 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:48:17.479517 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:48:17.479677 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:17.480048 2680 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:48:18.088435 2710 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:48:18.088699 2710 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:18.089434 2710 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:18.090629 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260929 20:48:18.092116 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:48:18.092378 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:48:18.092540 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:18.092828 2710 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:48:18.308177 388 master-stress-test.cc:578] Killing test thread 1/25
I20260929 20:48:18.656054 2714 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:48:18.656280 2714 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:18.656970 2714 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:18.658994 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:48:18.659255 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:48:18.659412 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:18.659647 2714 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:48:20.043120 2720 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:48:20.043375 2720 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:20.044054 2720 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:20.046494 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:48:20.046679 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:48:20.046797 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:20.047029 2720 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:48:21.413545 2726 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:48:21.413802 2726 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:21.414542 2726 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:21.417383 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:48:21.417627 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:21.417843 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:21.418201 2726 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:48:22.398321 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:48:22.398540 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:48:24.249955 2736 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:48:24.250190 2736 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:24.250864 2736 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:24.252170 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 7 similar messages]
W20260929 20:48:24.253465 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:48:24.253787 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:48:24.253959 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:24.254223 2736 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:48:26.281194 2742 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:48:26.281551 2742 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:26.282457 2742 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:26.285513 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:48:26.285738 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:26.285861 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:26.286178 2742 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:48:27.400116 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:48:27.400383 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:48:28.244859 2748 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:28.245086 2748 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:48:28.246196 2748 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:48:28.247236 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:48:28.247465 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:48:28.247677 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:48:28.247982 2748 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:48:30.484218 2750 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:30.484408 2750 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:48:30.485095 2750 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:48:30.486295 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:48:30.486539 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:48:30.486683 924 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:48:30.486945 2750 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:48:32.402143 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:48:32.402372 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:48:34.438388 2757 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:48:34.438671 2757 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:34.439499 2757 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:34.440747 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260929 20:48:34.442041 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:48:34.442296 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:34.442408 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:34.442787 2757 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:48:35.377607 2750 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:35.377806 2750 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:48:35.378422 2750 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:48:35.379390 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:48:35.379558 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:48:35.379679 925 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:48:35.380000 2750 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:48:36.210767 2750 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:36.210963 2750 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:48:36.211591 2750 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:48:36.212636 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:48:36.212860 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:48:36.212966 925 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:48:36.213339 2750 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:48:37.404158 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:48:37.404356 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:48:41.543397 2767 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:41.543622 2767 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:48:41.544312 2767 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:48:41.545198 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:48:41.545354 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:48:41.545497 925 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:48:41.545760 2767 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:48:42.406612 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:48:42.406872 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:48:42.822206 2770 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:48:42.822522 2770 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:42.823505 2770 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:42.824256 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:48:42.826128 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:48:42.826350 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:48:42.826534 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:42.826825 2770 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260929 20:48:45.047111 1037 heartbeater.cc:644] Failed to heartbeat to 127.0.97.62:46175 (55 consecutive failures): Network error: Failed to ping master at 127.0.97.62:46175: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:48:45.425058 1181 heartbeater.cc:644] Failed to heartbeat to 127.0.97.60:45599 (100 consecutive failures): Network error: Failed to ping master at 127.0.97.60:45599: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:48:45.436645 1323 heartbeater.cc:644] Failed to heartbeat to 127.0.97.60:45599 (100 consecutive failures): Network error: Failed to ping master at 127.0.97.60:45599: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:47.408432 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:48:47.408713 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:48:49.866675 2777 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:49.866888 2777 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:48:49.867854 2777 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:48:49.869020 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:48:49.869366 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:48:49.869518 924 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:48:49.869817 2777 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:48:52.410280 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:48:52.410467 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:48:53.090608 2786 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:48:53.090838 2786 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:53.091595 2786 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:53.093743 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:48:53.095324 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:48:53.095546 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:48:53.095703 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:53.095975 2786 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:48:54.887801 2783 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:48:54.888024 2783 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:48:54.888669 2783 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:48:54.889735 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:48:54.889909 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:48:54.890079 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:48:54.890383 2783 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:48:57.412182 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:48:57.412369 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:48:58.072119 2796 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:48:58.072297 2796 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:48:58.072954 2796 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:48:58.075349 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:48:58.075579 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:48:58.075742 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:48:58.076045 2796 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:49:02.414100 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:49:02.414294 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:49:03.767479 2811 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:49:03.767809 2811 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:49:03.768440 2811 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:49:03.773588 2488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260929 20:49:03.775161 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:49:03.775362 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:49:03.775555 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:49:03.775883 2811 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:49:07.417163 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:49:07.417415 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:49:09.457882 2818 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:49:09.458138 2818 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:49:09.459239 2818 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:49:09.460350 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:49:09.460573 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:49:09.460752 924 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:49:09.461035 2818 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:49:10.668164 2821 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:49:10.668380 2821 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:49:10.669011 2821 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:49:10.670066 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:49:10.670285 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:49:10.670452 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:49:10.670769 2821 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:49:12.419095 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:49:12.419355 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:49:17.421232 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:49:17.421434 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:49:17.656899 2827 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:49:17.657160 2827 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:49:17.657980 2827 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:49:17.659036 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:49:17.659245 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:49:17.659384 925 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:49:17.659726 2827 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:49:18.368185 2830 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:49:18.368453 2830 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:49:18.369266 2830 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:49:18.370450 2488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:49:18.371924 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:49:18.372121 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:49:18.372223 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:49:18.372421 2830 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:49:19.470492 2834 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:49:19.470696 2834 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:49:19.471469 2834 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:49:19.472586 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:49:19.472859 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:49:19.472956 925 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:49:19.473182 2834 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:49:22.423029 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:49:22.423311 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:49:24.522195 2834 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:49:24.522449 2834 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:49:24.523322 2834 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:49:24.524291 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:49:24.524492 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:49:24.524675 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:49:24.524899 2834 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:49:27.425056 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:49:27.425292 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:49:32.150931 2848 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:49:32.151165 2848 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:49:32.151820 2848 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:49:32.153255 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:49:32.154901 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:49:32.155165 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:49:32.155319 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:49:32.155683 2848 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:49:32.426957 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:49:32.427227 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:49:36.181774 2854 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:49:36.182006 2854 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:49:36.182842 2854 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:49:36.183825 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:49:36.184120 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:49:36.184283 925 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:49:36.184583 2854 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:49:37.429214 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:49:37.429441 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:49:39.463789 2858 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:49:39.464059 2858 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:49:39.464857 2858 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:49:39.465888 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:49:39.466045 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:49:39.466166 925 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:49:39.466396 2858 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:49:42.431599 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:49:42.431826 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 20:49:45.098560 1036 heartbeater.cc:644] Failed to heartbeat to 127.0.97.60:45599 (219 consecutive failures): Network error: Failed to ping master at 127.0.97.60:45599: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:49:45.344233 2860 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:49:45.344492 2860 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:49:45.345394 2860 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:49:45.346704 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:49:45.346941 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:49:45.347182 924 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:49:45.347452 2860 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260929 20:49:45.588672 1181 heartbeater.cc:644] Failed to heartbeat to 127.0.97.60:45599 (220 consecutive failures): Network error: Failed to ping master at 127.0.97.60:45599: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:49:45.600054 1323 heartbeater.cc:644] Failed to heartbeat to 127.0.97.60:45599 (220 consecutive failures): Network error: Failed to ping master at 127.0.97.60:45599: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:49:47.433369 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:49:47.433573 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:49:51.193284 2867 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:49:51.193557 2867 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:49:51.194473 2867 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:49:51.195654 2488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:49:51.197027 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:49:51.197256 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:49:51.197470 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:49:51.197755 2867 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:49:51.462527 2866 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:49:51.462751 2866 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:49:51.463572 2866 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:49:51.464564 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:49:51.464733 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:49:51.464869 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:49:51.465150 2866 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:49:52.435247 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:49:52.435413 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:49:57.437021 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:49:57.437196 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:49:58.205449 2878 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:49:58.205729 2878 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:49:58.206488 2878 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:49:58.207564 2488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:49:58.208750 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:49:58.208915 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:49:58.209026 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:49:58.209283 2878 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:49:59.103494 2883 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:49:59.103731 2883 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:49:59.104574 2883 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:49:59.105541 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:49:59.105773 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:49:59.105890 925 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:49:59.106184 2883 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:50:02.439030 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:50:02.439255 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:50:04.857870 2892 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:50:04.858174 2892 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:50:04.859112 2892 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:50:04.860363 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:50:04.862129 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:50:04.862323 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:50:04.862440 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:50:04.862715 2892 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:50:07.390836 2897 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:50:07.391083 2897 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:50:07.392485 2897 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:50:07.393836 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:50:07.394037 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:50:07.394294 924 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:50:07.394645 2897 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:50:07.441366 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:50:07.441617 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:50:10.439253 2906 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:50:10.439505 2906 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:50:10.440263 2906 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:50:10.441414 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:50:10.441614 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:50:10.441742 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:50:10.442003 2906 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:50:11.911458 2909 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:50:11.912146 2909 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:50:11.912966 2909 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:50:11.914090 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:50:11.915369 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:50:11.915594 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:50:11.915717 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:50:11.915984 2909 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:50:12.254544 2491 maintenance_manager.cc:419] P cc99fb1b059842088a50ba9c56fda6c1: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033360
I20260929 20:50:12.305470 2490 maintenance_manager.cc:643] P cc99fb1b059842088a50ba9c56fda6c1: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.050s user 0.026s sys 0.003s Metrics: {"bytes_written":13849,"cfile_init":1,"dirs.queue_time_us":497,"dirs.run_cpu_time_us":233,"dirs.run_wall_time_us":1057,"drs_written":1,"lbm_read_time_us":71,"lbm_reads_lt_1ms":4,"lbm_write_time_us":685,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":33,"thread_start_us":586,"threads_started":2,"wal-append.queue_time_us":383}
I20260929 20:50:12.306898 2491 maintenance_manager.cc:419] P cc99fb1b059842088a50ba9c56fda6c1: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1624 bytes on disk
I20260929 20:50:12.308005 2490 maintenance_manager.cc:643] P cc99fb1b059842088a50ba9c56fda6c1: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":150,"lbm_reads_lt_1ms":4}
I20260929 20:50:12.446812 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:50:12.447008 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:50:16.022169 2918 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:50:16.022367 2918 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:50:16.023192 2918 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:50:16.024642 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:50:16.024915 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:50:16.025081 924 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:50:16.025591 2918 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:50:17.448740 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:50:17.448958 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:50:22.450807 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:50:22.451025 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:50:26.746023 2940 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:50:26.746344 2940 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:50:26.747411 2940 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:50:26.748417 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:50:26.748587 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:50:26.748709 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:50:26.748926 2940 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:50:27.452405 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:50:27.452651 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:50:29.700621 2956 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:50:29.700865 2956 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:50:29.701798 2956 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:50:29.703078 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:50:29.704602 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:50:29.704850 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:50:29.704989 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:50:29.705255 2956 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:50:32.454452 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:50:32.454624 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:50:34.794112 2979 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:50:34.794333 2979 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:50:34.795181 2979 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:50:34.796321 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:50:34.796486 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:50:34.796720 925 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:50:34.796995 2979 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:50:35.040056 2979 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:50:35.040306 2979 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:50:35.041080 2979 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:50:35.041953 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:50:35.042107 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:50:35.042272 925 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:50:35.042528 2979 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:50:37.456480 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:50:37.456686 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:50:40.135437 3010 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:50:40.135640 3010 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:50:40.136232 3010 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:50:40.137976 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:50:40.139693 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:50:40.139915 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:50:40.140035 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:50:40.140269 3010 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:50:42.458961 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:50:42.459271 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 20:50:45.263021 1036 heartbeater.cc:644] Failed to heartbeat to 127.0.97.60:45599 (339 consecutive failures): Network error: Failed to ping master at 127.0.97.60:45599: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:50:45.760718 1181 heartbeater.cc:644] Failed to heartbeat to 127.0.97.60:45599 (340 consecutive failures): Network error: Failed to ping master at 127.0.97.60:45599: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:50:45.764568 1323 heartbeater.cc:644] Failed to heartbeat to 127.0.97.60:45599 (340 consecutive failures): Network error: Failed to ping master at 127.0.97.60:45599: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:50:47.462059 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:50:47.462333 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:50:49.730831 3053 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:50:49.731050 3053 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:50:49.732009 3053 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:50:49.732813 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:50:49.732975 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:50:49.733153 925 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:50:49.733421 3053 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:50:52.454978 3074 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:50:52.455346 3074 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:50:52.456087 3074 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:50:52.457056 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:50:52.457302 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:50:52.457432 924 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:50:52.457690 3074 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:50:52.464031 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:50:52.464218 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:50:55.637359 3092 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:50:55.637615 3092 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:50:55.638540 3092 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:50:55.639812 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:50:55.641225 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:50:55.641583 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:50:55.641743 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:50:55.642005 3092 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:50:56.246223 3091 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:50:56.246425 3091 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:50:56.247192 3091 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:50:56.248226 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:50:56.248413 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:50:56.248535 925 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:50:56.248782 3091 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:50:57.465865 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:50:57.466116 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:51:02.167209 3116 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:51:02.167464 3116 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:51:02.168344 3116 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:51:02.169301 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:51:02.169481 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:51:02.169607 925 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:51:02.169925 3116 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:51:02.467767 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:51:02.467967 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:51:03.814826 3133 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:51:03.815106 3133 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:51:03.815804 3133 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:51:03.817351 2488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:51:03.818585 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:51:03.818807 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:51:03.818971 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:51:03.819242 3133 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:51:07.470103 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:51:07.470361 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:51:09.509194 3163 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:51:09.509457 3163 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:51:09.510757 3163 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:51:09.511643 2488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:51:09.513064 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:51:09.513223 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:51:09.513356 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:51:09.513599 3163 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:51:09.774971 3162 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:51:09.775202 3162 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:51:09.776134 3162 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:51:09.777194 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:51:09.777385 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:51:09.777516 925 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:51:09.777778 3162 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:51:10.659674 3173 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:51:10.659961 3173 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:51:10.660787 3173 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:51:10.663254 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:51:10.663630 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:51:10.663832 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:51:10.664132 3173 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:51:12.472010 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:51:12.472225 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:51:14.903626 3200 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:51:14.903858 3200 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:51:14.904681 3200 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:51:14.905915 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260929 20:51:14.907847 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:51:14.908047 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:51:14.908180 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:51:14.908474 3200 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:51:16.947820 3195 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:51:16.948215 3195 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:51:16.949185 3195 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:51:16.950484 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:51:16.950717 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:51:16.951001 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:51:16.951435 3195 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:51:17.474572 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:51:17.474790 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:51:18.411798 3218 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:51:18.412009 3218 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:51:18.412959 3218 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:51:18.415700 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:51:18.416082 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:51:18.416247 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:51:18.416558 3218 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:51:19.992178 3215 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:51:19.992393 3215 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:51:19.993197 3215 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:51:19.994263 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:51:19.994490 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:51:19.994652 925 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:51:19.994994 3215 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:51:22.476773 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:51:22.477008 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:51:22.611274 3234 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:51:22.611564 3234 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:51:22.612517 3234 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:51:22.613473 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:51:22.613667 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:51:22.613840 925 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:51:22.614123 3234 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:51:24.122707 3248 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:51:24.123131 3248 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:51:24.123835 3248 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:51:24.125561 2488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260929 20:51:24.127149 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:51:24.127406 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:51:24.127640 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:51:24.127980 3248 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:51:27.478778 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:51:27.479070 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:51:32.480650 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:51:32.480823 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:51:33.067420 3290 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:51:33.067711 3290 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:51:33.068778 3290 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:51:33.070228 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:51:33.070474 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:51:33.070641 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:51:33.070909 3290 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:51:37.163875 3307 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:51:37.164094 3307 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:51:37.164990 3307 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:51:37.166116 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:51:37.166355 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:51:37.166472 925 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:51:37.166785 3307 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:51:37.482473 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:51:37.482713 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:51:38.917906 3323 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:51:38.918083 3323 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:51:38.918620 3323 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:51:38.919729 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:51:38.919945 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:51:38.920079 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:51:38.920373 3323 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:51:42.282480 3338 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:51:42.282683 3338 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:51:42.283535 3338 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:51:42.284682 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:51:42.285996 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:51:42.286203 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:51:42.286321 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:51:42.286607 3338 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:51:42.484086 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:51:42.484294 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 20:51:45.433447 1036 heartbeater.cc:644] Failed to heartbeat to 127.0.97.60:45599 (459 consecutive failures): Network error: Failed to ping master at 127.0.97.60:45599: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:51:45.934655 1323 heartbeater.cc:644] Failed to heartbeat to 127.0.97.60:45599 (460 consecutive failures): Network error: Failed to ping master at 127.0.97.60:45599: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:51:45.934715 1181 heartbeater.cc:644] Failed to heartbeat to 127.0.97.60:45599 (460 consecutive failures): Network error: Failed to ping master at 127.0.97.60:45599: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:51:47.485914 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:51:47.486164 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:51:49.200387 3371 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:51:49.200698 3371 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:51:49.201399 3371 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:51:49.202639 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:51:49.204094 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:51:49.204324 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:51:49.204528 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:51:49.204833 3371 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:51:49.804829 3370 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:51:49.805094 3370 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:51:49.805987 3370 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:51:49.807209 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:51:49.807436 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:51:49.807545 925 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:51:49.807770 3370 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:51:52.487877 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:51:52.488138 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:51:56.665376 3386 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:51:56.665601 3386 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:51:56.666400 3386 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:51:56.667467 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:51:56.667647 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:51:56.667769 924 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:51:56.667966 3386 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:51:57.036471 3409 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:51:57.036747 3409 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:51:57.037412 3409 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:51:57.038305 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:51:57.039991 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:51:57.040180 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:51:57.040374 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:51:57.040678 3409 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:51:57.489372 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:51:57.489566 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:52:02.491173 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:52:02.491333 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:52:04.534053 3442 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:52:04.534300 3442 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:52:04.535058 3442 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:52:04.536175 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:52:04.536446 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:52:04.536650 925 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:52:04.536988 3442 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:52:07.493924 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:52:07.494145 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:52:07.910703 3442 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:52:07.910903 3442 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:52:07.911859 3442 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:52:07.912917 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:52:07.913070 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:52:07.913204 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:52:07.913482 3442 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:52:10.584437 3468 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:52:10.584748 3468 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:52:10.585479 3468 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:52:10.589984 2488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:52:10.589984 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:52:10.591480 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:52:10.591678 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:52:10.591852 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:52:10.592133 3468 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:52:12.496196 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:52:12.496400 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:52:16.338253 3490 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:52:16.338522 3490 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:52:16.339476 3490 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:52:16.340525 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:52:16.340797 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:52:16.340996 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:52:16.341300 3490 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:52:17.498360 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:52:17.498636 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:52:18.223771 3490 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:52:18.223965 3490 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:52:18.224668 3490 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:52:18.225611 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:52:18.225833 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:52:18.225944 925 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:52:18.226269 3490 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:52:20.336565 3512 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:52:20.336803 3512 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:52:20.337568 3512 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:52:20.338689 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:52:20.340407 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:52:20.340658 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:52:20.340802 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:52:20.341094 3512 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:52:22.500370 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:52:22.500634 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:52:27.502321 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:52:27.502496 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:52:30.307152 3553 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:52:30.307379 3553 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:52:30.308346 3553 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:52:30.309335 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:52:30.309513 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:52:30.309829 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:52:30.310146 3553 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:52:31.879796 3561 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:52:31.880029 3561 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:52:31.880821 3561 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:52:31.882007 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:52:31.883687 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:52:31.883884 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:52:31.884048 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:52:31.884405 3561 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:52:32.504117 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:52:32.504354 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:52:36.144367 3577 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:52:36.144658 3577 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:52:36.145397 3577 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:52:36.146396 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:52:36.146579 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:52:36.146703 924 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:52:36.146960 3577 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:52:37.506052 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:52:37.506237 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:52:38.434026 3588 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:52:38.434455 3588 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:52:38.435971 3588 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:52:38.437135 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:52:38.438447 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:52:38.438675 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:52:38.438807 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:52:38.439018 3588 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:52:40.805095 3599 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:52:40.805296 3599 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:52:40.806120 3599 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:52:40.807066 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:52:40.807217 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:52:40.807353 925 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:52:40.808069 3599 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:52:42.185616 3608 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:52:42.185901 3608 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:52:42.186599 3608 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:52:42.188921 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:52:42.189134 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:52:42.189247 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:52:42.189450 3608 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:52:42.507494 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:52:42.507738 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:52:43.336530 3616 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:52:43.336810 3616 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:52:43.337529 3616 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:52:43.339910 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:52:43.340123 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:52:43.340231 2488 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:52:43.340513 3616 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:52:43.571643 3615 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:52:43.571913 3615 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:52:43.572736 3615 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:52:43.573719 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:52:43.574149 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:52:43.574247 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:52:43.574476 3615 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260929 20:52:45.600046 1036 heartbeater.cc:644] Failed to heartbeat to 127.0.97.60:45599 (579 consecutive failures): Network error: Failed to ping master at 127.0.97.60:45599: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:52:46.101542 1323 heartbeater.cc:644] Failed to heartbeat to 127.0.97.60:45599 (580 consecutive failures): Network error: Failed to ping master at 127.0.97.60:45599: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:52:46.101560 1181 heartbeater.cc:644] Failed to heartbeat to 127.0.97.60:45599 (580 consecutive failures): Network error: Failed to ping master at 127.0.97.60:45599: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
I20260929 20:52:47.509907 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:52:47.510124 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:52:52.511857 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:52:52.512041 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:52:57.513878 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:52:57.514073 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:53:00.668298 3685 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:53:00.668565 3685 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:53:00.669473 3685 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:53:00.670763 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:53:00.670935 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:53:00.671133 925 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:53:00.671375 3685 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:53:00.780797 3685 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:53:00.781005 3685 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:53:00.781669 3685 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:53:00.782421 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:53:00.782553 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:53:00.782718 924 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:53:00.782950 3685 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:53:01.526726 3689 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:53:01.527019 3689 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:53:01.527992 3689 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:53:01.529548 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260929 20:53:01.530831 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:53:01.530995 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:53:01.531116 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:53:01.531322 3689 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:53:02.515722 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:53:02.515949 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:53:04.969460 3685 raft_consensus.cc:482] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:53:04.969789 3685 raft_consensus.cc:504] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } } peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } }
I20260929 20:53:04.970525 3685 leader_election.cc:288] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081), 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835)
W20260929 20:53:04.971522 924 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
W20260929 20:53:04.971731 925 leader_election.cc:341] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: b2f7d9dde9ad4a10b7fbda43afa142c3
I20260929 20:53:04.971854 925 leader_election.cc:302] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:53:04.972206 3685 raft_consensus.cc:2726] T b2f7d9dde9ad4a10b7fbda43afa142c3 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:53:07.517840 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:53:07.518043 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:53:09.107559 2491 maintenance_manager.cc:419] P cc99fb1b059842088a50ba9c56fda6c1: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1624 bytes on disk
I20260929 20:53:09.113341 2490 maintenance_manager.cc:643] P cc99fb1b059842088a50ba9c56fda6c1: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.005s sys 0.000s Metrics: {"dirs.queue_time_us":459,"thread_start_us":555,"threads_started":2,"wal-append.queue_time_us":406}
I20260929 20:53:12.520104 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:53:12.520352 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:53:15.420408 3750 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:53:15.420780 3750 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:53:15.421506 3750 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:53:15.424271 2487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260929 20:53:15.426205 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:53:15.426473 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:53:15.426600 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:53:15.427156 3750 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260929 20:53:15.770941 3757 raft_consensus.cc:482] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 20:53:15.771271 3757 raft_consensus.cc:504] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb32e9690a546149d11339d371c832f" member_type: VOTER last_known_addr { host: "127.0.97.3" port: 32835 } } peers { permanent_uuid: "d1f79a0bcbdb42fe89e7812efeae1a86" member_type: VOTER last_known_addr { host: "127.0.97.1" port: 45657 } } peers { permanent_uuid: "cac815b9c12e40c6b63d4af82248f298" member_type: VOTER last_known_addr { host: "127.0.97.2" port: 41081 } }
I20260929 20:53:15.772187 3757 leader_election.cc:288] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835), cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081)
W20260929 20:53:15.773427 925 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cac815b9c12e40c6b63d4af82248f298 (127.0.97.2:41081): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
W20260929 20:53:15.773680 924 leader_election.cc:341] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb32e9690a546149d11339d371c832f (127.0.97.3:32835): Not found: Tablet not found: 2639596901a04bdb8bdbf8ccae3d1a50
I20260929 20:53:15.773880 924 leader_election.cc:302] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1f79a0bcbdb42fe89e7812efeae1a86; no voters: 1cb32e9690a546149d11339d371c832f, cac815b9c12e40c6b63d4af82248f298
I20260929 20:53:15.774214 3757 raft_consensus.cc:2726] T 2639596901a04bdb8bdbf8ccae3d1a50 P d1f79a0bcbdb42fe89e7812efeae1a86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 20:53:17.522503 2470 master_runner.cc:186] Error getting master registration for 127.0.97.62:46175: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111),
I20260929 20:53:17.522727 2470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 20:53:17.701159 3773 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b32aaf3821ff4ad09bcbfe11ff2e03c8)
I20260929 20:53:17.701447 3773 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32aaf3821ff4ad09bcbfe11ff2e03c8" member_type: VOTER last_known_addr { host: "127.0.97.62" port: 46175 } } peers { permanent_uuid: "cc99fb1b059842088a50ba9c56fda6c1" member_type: VOTER last_known_addr { host: "127.0.97.61" port: 34103 } } peers { permanent_uuid: "bfbad3ef4593415191fa0d993866d0ea" member_type: VOTER last_known_addr { host: "127.0.97.60" port: 45599 } }
I20260929 20:53:17.702284 3773 leader_election.cc:288] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175), bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599)
W20260929 20:53:17.704512 2488 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer bfbad3ef4593415191fa0d993866d0ea (127.0.97.60:45599): Network error: Client connection negotiation failed: client connection to 127.0.97.60:45599: connect: Connection refused (error 111)
W20260929 20:53:17.704754 2487 leader_election.cc:334] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b32aaf3821ff4ad09bcbfe11ff2e03c8 (127.0.97.62:46175): Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
I20260929 20:53:17.704942 2487 leader_election.cc:302] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc99fb1b059842088a50ba9c56fda6c1; no voters: b32aaf3821ff4ad09bcbfe11ff2e03c8, bfbad3ef4593415191fa0d993866d0ea
I20260929 20:53:17.705281 3773 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cc99fb1b059842088a50ba9c56fda6c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260929 20:53:18.083603 1333 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:53:18.084025 1333 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:53:18.084203 1333 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
W20260929 20:53:18.084368 1333 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
F20260929 20:53:18.095419 1354 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
*** Aborted at 1790715198 (unix time) try "date -d @1790715198" if you are using GNU date ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status:
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098807 1349 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098807 1349 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098999 1347 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a988b1827f9e482086b9b9ad0663c2b5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098807 1349 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098999 1347 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a988b1827f9e482086b9b9ad0663c2b5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099434 1359 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098807 1349 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098999 1347 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a988b1827f9e482086b9b9ad0663c2b5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099434 1359 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099673 1342 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0d85111ed1f34848b4c76ba1a254e00f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098807 1349 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098999 1347 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a988b1827f9e482086b9b9ad0663c2b5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099434 1359 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099673 1342 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0d85111ed1f34848b4c76ba1a254e00f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099877 1344 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_be3c404093574a04aea5256b4ecba3bb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098807 1349 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098999 1347 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a988b1827f9e482086b9b9ad0663c2b5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099434 1359 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099673 1342 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0d85111ed1f34848b4c76ba1a254e00f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099877 1344 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_be3c404093574a04aea5256b4ecba3bb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101038 1348 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_57d2f2bbf5c3465ba1591d6a2b6fbbc0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098807 1349 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098999 1347 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a988b1827f9e482086b9b9ad0663c2b5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099434 1359 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099673 1342 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0d85111ed1f34848b4c76ba1a254e00f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099877 1344 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_be3c404093574a04aea5256b4ecba3bb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101038 1348 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_57d2f2bbf5c3465ba1591d6a2b6fbbc0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101147 1355 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098807 1349 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098999 1347 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a988b1827f9e482086b9b9ad0663c2b5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099434 1359 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099673 1342 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0d85111ed1f34848b4c76ba1a254e00f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099877 1344 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_be3c404093574a04aea5256b4ecba3bb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101038 1348 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_57d2f2bbf5c3465ba1591d6a2b6fbbc0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101147 1355 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101362 1340 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_031408b68a614af6b6ef8a7fe493dd75 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101470 1360 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098807 1349 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098999 1347 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a988b1827f9e482086b9b9ad0663c2b5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099434 1359 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099673 1342 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0d85111ed1f34848b4c76ba1a254e00f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099877 1344 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_be3c404093574a04aea5256b4ecba3bb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101038 1348 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_57d2f2bbf5c3465ba1591d6a2b6fbbc0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101147 1355 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101362 1340 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_031408b68a614af6b6ef8a7fe493dd75 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101470 1360 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098807 1349 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098999 1347 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a988b1827f9e482086b9b9ad0663c2b5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099434 1359 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099673 1342 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0d85111ed1f34848b4c76ba1a254e00f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099877 1344 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_be3c404093574a04aea5256b4ecba3bb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101038 1348 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_57d2f2bbf5c3465ba1591d6a2b6fbbc0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101147 1355 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101362 1340 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_031408b68a614af6b6ef8a7fe493dd75 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101470 1360 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101583 1341 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_55d763f6ec744322a43d80efeb51983e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098807 1349 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098999 1347 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a988b1827f9e482086b9b9ad0663c2b5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099434 1359 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099673 1342 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0d85111ed1f34848b4c76ba1a254e00f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099877 1344 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_be3c404093574a04aea5256b4ecba3bb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101038 1348 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_57d2f2bbf5c3465ba1591d6a2b6fbbc0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101147 1355 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101362 1340 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_031408b68a614af6b6ef8a7fe493dd75 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101470 1360 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101583 1341 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_55d763f6ec744322a43d80efeb51983e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101166 1346 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e3c4d53b991348e8af5e335ad5847221 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098807 1349 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098999 1347 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a988b1827f9e482086b9b9ad0663c2b5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099434 1359 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099673 1342 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0d85111ed1f34848b4c76ba1a254e00f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099877 1344 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_be3c404093574a04aea5256b4ecba3bb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101038 1348 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_57d2f2bbf5c3465ba1591d6a2b6fbbc0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101147 1355 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101362 1340 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_031408b68a614af6b6ef8a7fe493dd75 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101470 1360 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101583 1341 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_55d763f6ec744322a43d80efeb51983e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101166 1346 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e3c4d53b991348e8af5e335ad5847221 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101718 1350 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098807 1349 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098999 1347 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a988b1827f9e482086b9b9ad0663c2b5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099434 1359 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099673 1342 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0d85111ed1f34848b4c76ba1a254e00f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099877 1344 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_be3c404093574a04aea5256b4ecba3bb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101038 1348 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_57d2f2bbf5c3465ba1591d6a2b6fbbc0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101147 1355 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101362 1340 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_031408b68a614af6b6ef8a7fe493dd75 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101470 1360 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101583 1341 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_55d763f6ec744322a43d80efeb51983e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101166 1346 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e3c4d53b991348e8af5e335ad5847221 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101718 1350 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.102116 1345 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_21a7c2dce9cf486580fad3131b93aa63 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098807 1349 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098999 1347 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a988b1827f9e482086b9b9ad0663c2b5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099434 1359 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099673 1342 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0d85111ed1f34848b4c76ba1a254e00f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099877 1344 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_be3c404093574a04aea5256b4ecba3bb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101038 1348 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_57d2f2bbf5c3465ba1591d6a2b6fbbc0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101147 1355 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101362 1340 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_031408b68a614af6b6ef8a7fe493dd75 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101470 1360 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101583 1341 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_55d763f6ec744322a43d80efeb51983e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101166 1346 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e3c4d53b991348e8af5e335ad5847221 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101718 1350 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.102116 1345 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_21a7c2dce9cf486580fad3131b93aa63 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.103245 1364 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098807 1349 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098999 1347 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a988b1827f9e482086b9b9ad0663c2b5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099434 1359 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099673 1342 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0d85111ed1f34848b4c76ba1a254e00f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099877 1344 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_be3c404093574a04aea5256b4ecba3bb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101038 1348 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_57d2f2bbf5c3465ba1591d6a2b6fbbc0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101147 1355 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101362 1340 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_031408b68a614af6b6ef8a7fe493dd75 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101470 1360 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101583 1341 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_55d763f6ec744322a43d80efeb51983e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101166 1346 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e3c4d53b991348e8af5e335ad5847221 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101718 1350 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.102116 1345 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_21a7c2dce9cf486580fad3131b93aa63 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.103245 1364 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.103348 1365 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098807 1349 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098999 1347 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a988b1827f9e482086b9b9ad0663c2b5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099434 1359 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099673 1342 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0d85111ed1f34848b4c76ba1a254e00f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099877 1344 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_be3c404093574a04aea5256b4ecba3bb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101038 1348 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_57d2f2bbf5c3465ba1591d6a2b6fbbc0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101147 1355 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101362 1340 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_031408b68a614af6b6ef8a7fe493dd75 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101470 1360 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101583 1341 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_55d763f6ec744322a43d80efeb51983e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101166 1346 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e3c4d53b991348e8af5e335ad5847221 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101718 1350 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.102116 1345 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_21a7c2dce9cf486580fad3131b93aa63 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.103245 1364 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.103348 1365 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.103446 1357 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260929 20:53:18.096850 1363 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097003 1356 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097484 1343 master-stress-test.cc:329] Check failed: _s.ok() Bad status: F20260929 20:53:18.097499 1353 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e1a3cd10e94d42c598e9c071db2144d7 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.097506 1351 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098663 1352 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098807 1349 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.098999 1347 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a988b1827f9e482086b9b9ad0663c2b5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099434 1359 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099673 1342 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0d85111ed1f34848b4c76ba1a254e00f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.099877 1344 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_be3c404093574a04aea5256b4ecba3bb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101038 1348 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_57d2f2bbf5c3465ba1591d6a2b6fbbc0 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101147 1355 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101362 1340 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_031408b68a614af6b6ef8a7fe493dd75 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101470 1360 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101583 1341 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_55d763f6ec744322a43d80efeb51983e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101166 1346 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e3c4d53b991348e8af5e335ad5847221 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.101718 1350 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.102116 1345 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_21a7c2dce9cf486580fad3131b93aa63 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.103245 1364 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.103348 1365 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.103446 1357 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)F20260929 20:53:18.104085 1339 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dae8faea7e2b44909906d91d85898732 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.97.62:46175,127.0.97.61:34103,127.0.97.60:45599, num_attempts: 770) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.97.62:46175: connect: Connection refused (error 111)
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800000184) received by PID 388 (TID 0x7fee04ed5700) from PID 388; stack trace: ***
@ 0x7fee1a4a5a46 google::(anonymous namespace)::FailureSignalHandler() at ??:0
@ 0x7fee1e5e7980 (unknown) at ??:0
@ 0x7fee177d3e87 gsignal at ??:0
@ 0x7fee177d57f1 abort at ??:0
@ 0x7fee1a4979ad google::LogMessage::Fail() at ??:0
@ 0x7fee1a49a88a google::LogMessage::SendToLog() at ??:0
@ 0x7fee1a4974de google::LogMessage::Flush() at ??:0
@ 0x7fee1a498a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x405273 kudu::MasterStressTest::DeleteTableThread() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/master-stress-test.cc:352
@ 0x444b44 std::__invoke_impl<>() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/bits/invoke.h:73
@ 0x4449fd std::__invoke<>() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/bits/invoke.h:95
@ 0x4449ba _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:244
@ 0x44496e std::thread::_Invoker<>::operator()() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:253
@ 0x4446a9 std::thread::_State_impl<>::_M_run() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:196
@ 0x7fee22f23a4f execute_native_thread_routine at ??:0
@ 0x7fee1e5dc6db start_thread at ??:0
@ 0x7fee178b661f clone at ??:0